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CHAPTER I. 

INTRODUCTION: HOW TO USE THIS BOOK 


In intelligence production as yet we find little study of 
methods as such. Yet a systematic study of methods in 
any given field . . . nearly always leads to.. . improve¬ 
ment. 


Washington Platt 
1957 


Purpose 

This work was prepared for the students of the Defense 
Intelligence School who may be required to demonstrate their 
knowledge and skill in conducting their own research projects. 
It is conceivable that analysts and researchers on the job might 
find sections of the book useful, but the book was not prepared 
for experienced intelligence analysts or researchers. Above all, 
this book should not be construed as a text for training 
intelligence analysts, even though analytic processes and prob¬ 
lems are described. The purpose of the text is to provide 
guidance to the inexperienced researcher who must thread his 
way through a tortuous maze of procedures and steps as he 
defines his research problem, collects his information, analyzes 
it, and prepares his report. 

Although the book was prepared essentially for the candi¬ 
date for the Master of Science degree in Strategic Intelligence, 
other students of the Defense Intelligence School who may be 
unfamiliar with the peculiarities of intelligence research may 
also find portions of the text useful. Numerous books have been 
written on research methodologies appropriate for various 
disciplines—history, education, psychology, anthropology, and 
so on. This book was prepared to fill a gap, namely, to describe 
research procedures and representative techniques specifically 
related to intelligence. 
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The Intended Reader 


Students attending the Defense Intelligence School reflect a 
wide variety of academic backgrounds, prior assignments, skills, 
and service experience. For example, some students may have 
been in intelligence billets prior to their assignment to the 
School. For others, assignment to the School may be their first 
formal exposure to intelligence. As such, the book was prepared 
for a diverse body of readers. 

Preparing a single book for a diverse body of readers poses 
problems. Obviously, not all of the content will be highly 
relevant to all of the readers, nor will all of the content be 
comprehended easily by all of the students. Therefore, it is 
suggested that the reader use the text selectively. Although the 
sequence of research phases is fairly ridigly established and 
pertinent to all kinds of research activities, the methodologies 
described in the later chapters should be examined primarily 
with an eye to possible application to the researcher’s problem. 
The intent of the sections dealing with methodologies is not to 
train professional “methodologists.” Instead, the intent is to 
familiarize the researcher with techniques that have been and 
are being used in intelligence research and analysis. 

Readers vary not only in experience, but also in personal 
preferences and inclinations. Reading may be a relatively 
enjoyable task or pastime to some, but an onerous, time- 
consuming task to others. Regardless of personal proclivities, 
however, it is recognized that time is valuable to the reader and 
nonreader alike. Therefore, the purpose of this introductory 
chapter is to help the reader utilize his reading time most 
efficiently. 

For example, students with much intelligence experience 
would probably need not read Chapter II, “Setting the Stage: 
An Overview of the Evolution of Strategic Intelligence Produc¬ 
tion,” and Chapter III, “Inaccessible and Questionable Informa¬ 
tion: The Need for Intelligence.” Readers with advanced 
degrees from other institutions who have also had much 
experience in intelligence, could probably skip Chapter IV, 
“Research: A Description of the Activity,” Chapter V, 


“Intelligence Research/Academic Research: Similarities and 
Differences,” and Chapter VI, “Characteristics of the Intelli¬ 
gence Researcher,” as well. 

Many readers dislike theoretical discussions. Sometimes 
they are necessary, however. It is strongly recommended that all 
students read Chapter VII, “Types of Inquiry and the Nature of 
Proof,” and Chapter VIII, “The Relation of Induction and 
Deduction to Theory Building in Intelligence Research, despite 
the fact that the chapters are “theoretical.” 

Organization and Content 

From Chapter IX to the end of the book, the organization 
of the text follows the normal sequence of phases in performing 
research, from the planning phase to the preparation of the final 
report. There are major digresssions, however, particularly in 
the discussion of analysis, in which specific practices are 
considered in detail. 

Major portions of the chapters relating to the analysis phase 
of a research project deal with specific methodologies. An 
attempt has been made to describe the various methodologies, 
and to show where and how they might be applied in 
intelligence research. However, since many of the methodo¬ 
logies would require volumes to describe in detail, this book 
cannot be considered a how-to-do-it manual for these methods. 
For detailed information on any specific methodology, the 
reader is advised to consult the works referenced in the various 
sections. 

Obviously, not every methodology used in intelligence 
analysis or research has been described in this book. Many 
techniques are highly specialized and would require months of 
training before the researcher could apply them. Examples of 
these techniques would be photo and image interpretation and 
signal analysis. Psychoanalytic techniques referred to in the 
methodology sections would also require years of training. 
Therefore, techniques which the “average student ot t e 
Defense Intelligence School might employ have been stressed. 
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Security considerations prevented the relating of certain 
techniques to specific situations. Here the imagination of the 
reader will be required to see exactly where and how a 
technique could have been used in a specific context. For 
example, one of the more universal analytic approaches involves 
the use of analogies. In instances in which the researcher is 
denied access to certain real-world data, he may have no other 
recourse than to fall back on analogous situations and infer 
from these. The potential fallacies of the approach are 
numerous, but, as discussed in Chapter V, intelligence re¬ 
searchers are required to arrive at conclusions despite the 
unavailability of reliable data. Where analogies are used in 
addressing current intelligence problems is not indicated in this 
text, but the reader can assume that the practice is universal. 

Many examples cited in the text are genuine. However, for 
purposes of security, contexts may have been changed, and the 
perennial “other country” may have been substituted for a real 
name. 

Any work attempting to encompass a topic as large as 
intelligence research methodology must, by necessity, exclude 
certain content. In this book, for example, details on report 
preparation—details relating to footnoting, forms and formats— 
have been excluded deliberately. This does not mean that these 
details are unimportant: but these details are treated much 
better in the referenced works that are devoted specifically to 
these topics. 


4 


CHAPTER II. 

SETTING THE STAGE: AN OVERVIEW 
OF THE EVOLUTION OF STRATEGIC 
INTELLIGENCE PRODUCTION 


If you know the enemy and know yourself, you need 
not fear the results of a hundred battles. If you know 
yourself but not the enemy, for every victory you will 
suffer a defeat. If you know neither yourself nor the 
enemy, you are a fool and will meet defeat in every 
battle. 

Sun Tzu 


Intelligence research is an activity most commonly associ¬ 
ated with strategic intelligence production. Strategic intelligence 
in the sense that it is known today is a fairly recent innovation 
which evolved from its predecessor, tactical (or combat, or 
“military”) intelligence. This section traces briefly the evolution 
of strategic intelligence. The thesis of this overview is that as 
methods of resolving conflict became more complex so did 
strategic intelligence production become more complex. 

Organized Warfare and the Need 
for Intelligence 

Organized warfare gave rise to the need for military and 
strategic intelligence. When conflict had to be resolved on a 

1 Dozens of books have been written since the end of World War II 
that trace the history of intelligence operations or production, or that 
describe the origin of a particular intelligence organization. Few books, 
however, surpass Richard Wilmer Rowan’s The Story of Secret Service 
(Garden City, New York: Garden City Publishing Co., Inc., 1939). 
Admittedly dated, this book is not only scholarly, but also entertaining: a 
masterpiece of its kind. 
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man-to-man basis there was little need for intelligence. The 
principals knew each other and the field of battle was familiar 
to both combatants. When conflicts involved tribes, and later, 
when conflicts involved armies, conditions changed. Although 
combatants may have known who their adversaries were, they 
needed additional information. The necessity to move large 
groups of soldiers quickly and efficiently made certain types of 
knowledge imperative—knowledge of the terrain, location of the 
enemy, his strength, and his deployment. Apparently, it was 
recognized very early that the average soldier was not a suitable 
agent for procuring this information. Skillful, perhaps, in 
handling his weapon and formidable in endurance, the average 
soldier was not able to insinuate himself into the enemy ranks, 
was not able to estimate well, probably was not able to read nor 
record, and was hardly able to process mentally all of the 
needed information relating to the ability of an opposing force 
to do battle. Therefore, it is not surprising that early intelli¬ 
gence “systems” were built around spies and agents—those 
specialists who, either by training or by inclination, understood 
what kinds of information were significant, and were suffi¬ 
ciently cunning or deceitful to obtain it. 

Agents and spies have always been a part of any covert 
intelligence system and will probably continue to be so. For 
example, prior to his campaign against Rome in the third 
century B.C., Hannibal infiltrated his agents into northern Italy, 
and undoubtedly one of their more essential elements of 
information related to passes through which Hannibal could 
move his cavalry and elephants. 2 Similarly, Genghis Khan in the 
thirteenth century used scouts for ground reconnaissance, but 
he also used well-placed agents for spreading rumors of his 
terror and intrepidity. Obviously, the Khan’s systematic 
employment of rumor as a terror weapon exploited someone 
else’s intelligence system, as unstructured as it may have been. 

2 For the researcher with a bent for “historical” intelligence analysis, 
determining the source of Hannibal’s elephants might provide a stimulating 
diversion. Did the elephants come from India or from Africa? If they came 
from India, how did they make the trek westward? If they came from 
Africa, where were the herds located? 
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But it is significant to note that even as early as the Middle 
Kingdom of Egypt (1580-1150 B.C.), the necessity to obtain 
intelligence on a regular and systematic basis was Perceived 
and units were especially trained and employed to collect it. 
The Hyksos in Egypt, according to Field Marshal Bernard 
Montgomery, had an unusually effective intelligence orgamza 
tion which utilized a variety of communication media from 
torches to runners and employed special reconnaissance units tc 
scout terrain and capture prisoners for interrogation Inter¬ 
estingly, reports sent by officers to their superiors at head¬ 
quarters had to indicate the source of their information a 
practice which, with variations is still practiced today by the 
US. intelligence community. 3 Tins is one of the earl 
indications of systematic procedures for obtaining mtelhgenc , 
but according to the Old Testament, there were procedures 
which even predated the Hyksos. 

Sun Tzu: First Among Few Writers 
of a Very Old Profession ~ 

Aside from Biblical references, the first writer to document 
the importance of evaluated information about a real or 
potential enemy was Sun Tzu in 500 B.C. What enabled the wise 
sovereign and the good general to achieve thl "S s be *° nd f 
reach of ordinary men, according to Sun Tzu, was tore 
knowledge. 4 Clearly, today technology has changed the nature 
of the process of developing this foreknowledge and the 
assertion by Sun Tzu that knowledge of the enemy s disposition 
can be obtained only from other men (spies) is indeed quaint in 
an era in which major powers employ satellites, where few 
secrets are held for long in the “ether”, and where practically 
every emanation of the “other side”-heat, light RF energy, 
seismic waves, and so on- are potential sources of intelligence 

3 Field Marshal Viscount Bernard Montgomery,^ History of Warfare 
(Cleveland: The World Publishing Company, 1968), p. 39. 


4 Thomas R. Phillips, ed., Roots of Strategy (Harrisburg, Pennsylvania. 
The Military Service Publishing Company, 1955). Sun Tzu On the Art of 
War, by Lionel Giles, trans., pp. 60-1. 
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information. What is significant about Sun Tzu is not that he 
recognized the importance of intelligence, but that he recorded 
his maxims on the collection and evaluation of intelligence 
information. Sun Tzu’s Art of War influenced Chinese and 
Japanese military thinking for over 2400 years, and even in 
recent times this classic was required reading for all of Mao 
Ise-tung s lieutenants. 5 

From the time of Sun Tzu to the Middle Ages, there is 
increasing evidence of intelligence as a necessary concomitant of 
warfare. Alexander’s empire could not have survived as long as 
it did without Alexander’s lieutenants possessing a knowledge 
and understanding of the customs of the conquered people. 

Mithradates, a one-man intelligence staff and an intellectual 
genius who mastered twenty-two languages, collected his basic 
intelligence firsthand by wandering on foot throughout Asia 
Minor During the first century B.C., his army constituted the 

t j lr ? a 1 t to R oman hegemony in the Mediterranean. 
Mithradates knowledge of the external enemy was superior to 
his knowledge of the internal enemy, however, because he was 
unable to anticipate a mutiny in his own ranks. 

Hannibal’s employment of agents as part of his campaign 
against Rome was mentioned earlier; and Scipio, the nemesis of 
Hannibal was equally adept at using information about the 
enemy s disposition to his own advantage. 

The Roman gerund Suetonius Paulinus who, with 10,000 
men defeated 230,000 Britons in 62 A.D., had been a fonner 
intelligence officer in Africa in 47 A.D. His descriptions of the 


Allen Dulles, The Craft of Intelligence (New York: Harper & Row 
Publishers, 1963), p. 13. 


terrain and habitants—presumably intelligence studies—still 
survive.^ 

Other than recorded instances of success or failure due to 
surprise (the very thing intelligence is intended to avert), very 
little has been recorded about early intelligence doctrine, 
methods, or procedures. One of the earliest examples of 
intelligence doctrine, however, is found in Vegetius’ Military 
Institutions of the Romans (ca. 400 A.D.), and the tenets set 
forth about the importance of information of terrain, avenues 
of approach, and routes of march are still relevant today. 

From the Middle Ages to the Renaissance little was 
recorded about the production or utilization of intelligence. 
This is strange, because significant and decisive events occurred 
during the period, for example: the battles of Tours and 
Hastings, the Crusades, the intercontinental campaigns of 
Genghis Khan and later Kublai Khan, and the battle of Crecy. 
From what details exist of the conflicts it would appear that 
intelligence played a very small role in determining the 
outcomes of these events. 

With the exception of Genghis Khan, commanders 
apparently paid little attention to collecting or utilizing 
intelligence in any systematic way. Edward III, for example, 
had so little information about the terrain prior to the battle of 
Crecy that he had to bribe a local French peasant to guide his 
army to a ford in the Somme. 

Emphasis on Internal Security 

versus External Threat 

In the East, internal security was the dominant focus of all 
intelligence activities. During the Edo period in Japan, for 

6jt is interesting to note how many prominent military leaders spent 
at least a part of their careers in intelligence. Aside from Mithradates and 
Suetonius Paulinus, of course, there are Gordon of Khartoum, T. E. 
Lawrence, Orde Wingate, and Joseph W. Stilwell (as a military attache in 
Peking). All of these officers ultimately distinguished themselves as 
military leaders in those countries where they had served in some 
intelligence capacity. 
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l a :- Ple ’i he Z et (spies ’ Censors ’ and agents provocateurs) 
monitored activities of dissident elements; 7 and earlier, the 

ninja perhaps the most versatile agents of all times-were 
trained and utilized in the martial arts of scouting and 

SZTT* and i 1 " the f COV l ert arts of political assassination. 8 
though the exploits of the ninja have been depicted in 

numerous Japanese motion pictures and television productions 

WhaMsT i thCSe tal6S 3re apoohryphal is unknown’ 
What is known, however, is that the evolution of the art of 
ninjutsu was influenced by the writings of Sun Tzu. 

Likewise in the West, at least until the seventeenth century 
ore effort seemed to be devoted to internal security than to a 
systematic analysis of another country’s capabilities and 
vulnerabilities In part, this was a function of the spirit of the 

,!“ T WhlCh C ° Urt lntrigue constituted perhaps more of a 
threat to a reigning monarch than did some foreign power 
ence, Elizabeth had her Walsingham and Cromwell his Thurloe.’ 

Systematic Strategic Intelligence 
Production Ignored: Some Hypotheses 

Several hypotheses might be postulated to account for why 
relatively little emphasis was placed upon the systematic 

surnri, 1 ° f f a : egiC ,ntelll8ence - Rrst, the element of 
surprise was difficult to maintain. Once a decision had been 

made to engage in an act of war, the signs were fairly obvious. 

or example, in 1587 it was common knowledge (at least to 
those to whom the information was important) that an armada 
would be dispatched from Spain. Admittedly, exactly when and 
from which ports an armada would sail were unknown initially 

h S l nh att ? r ° f lnterest > the timel y raid on Cadiz by Drake set 
back the departure of the Spanish Armada by at least a year, 

d even Philip II did not know exactly when the Armada 
would be ready. When the Armada d id set forth (after a series 

jEdwin 0. Reischauer and John K. Fairbank, East Asia: The Great 
Tradition (Boston: Houghton Mifflin Company, 1960), p. 607. 


S Donn F. Draeger and Robert W. Smith, Asian Fighting Arts (Tokyo- 
Kodansha International, Ltd., 1969), p. 120 ff. 1 y ' 
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of storms which further delayed the departure), picket boats 
quickly alerted the English who had prepared for months for 
this engagement. 

Another reason for the relatively little emphasis placed 
upon the production of strategic intelligence was the manner in 
which warfare was conducted. Military campaigns usually 
involved one or a series of set piece battles in which the 
personalities of the leaders—their audacity, tactics, modes of 
employing infantry, cavalry, and artillery—were known to the 
key participants. 

Not only were personalities known to the opposing 
combatants, but also the tactics employed by all 
combatants were generally the same. Interestingly, when 
innovations were introduced, the results were indeed disastrous 
for the opposing side. For example, the Spanish intended to 
employ the Armada against the English in the same manner as 
they engaged the Turks in the Battle of Lepanto. Warships 
would make physical contact, and the infantry would board the 
English ships, clear the decks, and capture the ships. However, 
the English engaged the enemy by cannon and refused to close 
with the Spanish men-of-war. Thus, the tactic that worked so 
well in the past was useless against an innovative opponent. 

Gustavus Adolphus of Sweden was an outstanding military 
innovator. He drilled his cannoneers to such a degree of 
perfection that his artillery could fire three rounds to the 
enemy’s one. The high rate of fire combined with mobility 
made his artillery a terror weapon in the Thirty Years War. 
Gustavus was innovative in the field of intelligence as well. His 
addition of a “Chief of Scouts” to his staff predated by nearly a 
hundred years similar intelligence staffs in the French and 
German armies. 

Still another reason why relatively little emphasis was 
placed on the production of strategic intelligence was that in 
the period between the Middle Ages and the mid-Renaissance 
conflicts were not global; hence, the necessity for large amounts 
of detailed information about terrain (other than routes of 
march), customs of the people, and so on, was not perceived to 
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to 0f a Cam P ai 8"- °" e stable exception 

giooai conflicts, of course, were the Crusariec 

one cannot but be amazed at the audacity (or arrogance) of the 

s ren,r S r efS - Wh °’ ^ little kn °wledge of ?he enemy’ 
strength terram, routes of march, or logistics, nevertheless 
ndertook a campaign that crossed a continent. 

It should be stated that there may have been instances in 

2jjJ. C 5 Str . ateglC intel |igence was produced systematically on a 
global scale prior to the sixteenth century. Such systems if thev 

of Their ^ certainl y not publicized and the extent and nature 
of their operations are unknown, 

the mlr WOUld n °} necessaril y expect to find the details of 
,Tnn r K Ce production activities publicized. Battlefield 

tion bm n inteir ame COmmon kn °wledge after their introduc- 
tion, but intelligence innovations, if they did exist probablv 

ucc e ed ed because little was known of them by The opposing 
side. Elizabeth, for example, would have been eager to have her 
court chroniclers document the successes of her reign fire New 
World explorations, the exploits of the privateerf (after shl 

Snl£ly W1 that sh lllX “m ° f the Arm ada-but it is 

unlikely that she would have permitted anyone to document 

her agents’ penetration of the court of Tuscany. aocument 

Precursors to the "M ortem Era" in 
Strategic Intelligence Production 

be IlJ a ,T\ P “ r ‘ ° f ,he / ifte “' h ^ntury marks what might 
be called the beginning of the “modern era” in strategic and 

military intelligence production. This was the beginning^ the 

dyn»k O rut eX am ra,i0 h ^ ' he begi " ninS ° f the declilK ° f 
f rule ’ although dynastic rule would continue for 

in which 1V thp Ce t nt f, ne T Tlm W3S alS ° the beginning of the period 
ch the technology and techniques of warfare began to 

oft: fit mode StaVUS Ad ° lphUS ’ menti ° ned earIier ’ -s on2 

ot the first modem innovators, and others followed such as 
Frederick the Great, and later Marlborough and Napoleon. 

Like Vegetius more than a thousand years earlier tacticians 
an strategists again began to formulate doctrine, and, signifi- 
antly, military intelligence was an important element of this 


doctrine. Admittedly dated, and perhaps quaint, the early 
precepts are nevertheless still timely. For example, writing 
primarily for Prussian officers, Frederick the Great fully 
appreciated the importance of information relating to the 
enemy’s country, and his exhortation that “knowledge of the 
country is to a general what a rifle is to an infantryman and the 
rules of arithmetic are to the geometrician,” is as pertinent 
today as it was in the mid-eighteenth century. 9 

Maurice de Saxe, who gained firsthand experience under 
Marlborough and Eugene of Savoy, stated in his Reveries some 
general rules for the interpretation of “signs”—those physical 
manifestations of battle which, if interpreted correctly, give 
some indication of the opposing army’s intent. Although the 
interpretations of specific signs may no longer be pertinent, the 
act of drawing reliable inferences from commonplace signs is a 
procedure that is very relevant today to intelligence analysts 
and researchers. 10 

Despite the paucity of information relating to a govern¬ 
ment’s systematic production of strategic intelligence prior to 
the seventeenth century, there is documentation attesting to 
other institutions’ use (and production of) strategic intelligence. 
For example, one of the earlier instances of a non-governmental 
agency’s strategic intelligence activities were those carried out 
by the various banking houses in Europe in the mid-sixteenth 
century. 


9 Thomas R. Phillips, trans., Frederick the Great , Instructions for His 
Generals (Harrisburg, Pennsylvania: The Military Service Publishing Com¬ 
pany, 1944), p. 47. 


^Saxe wrote that when you hear much firing from the enemy camp, 
you may expect an engagement the day following because the men are 
discharging and cleaning their weapons. My Reveries Upon the Art of War 
by Marshal Maurice de Saxe in Phillips, Roots of Strategy, p. 292. ff. This 
is reminiscent of the WW II practice of Allied aircraft test firing their 
weapons shortly after takeoff and the standing operating procedure for 
U.S. forces in Vietnam to discharge the weapons in a perimeter defense 
shortly after daybreak. 
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But even prior to the activities of the large banking houses 
were the “strategic intelligence” activities carried out by 
Western traders and merchants in the Middle Ages. To these 
vendors, economic information, customs and practices, routes 
of communications, and political information relating to 
“target” Eastern countries and populations were critical to the 
success or failure of their enterprises. The extent to which 
merchants shared this information about their customers is 
unknown. But it is known that very little systematic use of this 
information was made by governments—a monumental blunder 
at a time when the East constituted the major strategic threat to 
all of Western Europe. 

U.S. Intelligence Organizations: 

Newcomers to an Old Club 

Compared with England, France, Spain, and Portugal, the 
United States is a relative newcomer to the international arena 
of power and politics; therefore, it is not surprising that the 
United States’ history of strategic intelligence production is a 
short one. Although scouts and agents (and even detectives 11 ) 
had been used to gather and evaluate intelligence about the 
opposing sides during the earlier conflicts in America, it was not 
until World War I that the necessity for producing strategic 
intelligence on a global scale became evident. 

Having only a skeletal intelligence organization at the 
outbreak of World War I, the U.S. was forced to rely heavily 
upon the intelligence produced by the British and French. 
Although lessons were learned quickly and a cadre of intelli¬ 
gence personnel was trained, little remained of this cadre other 
than the organizational structure, during the interim between 
World War I and World War II. 

In World War II, the U.S. again had to create a large-scale 
strategic intelligence production capability, and again the 
Americans learned much from their British counterparts. By the 


1 President Lincoln employed the Pinkerton Detective Agency to 
conduct the Union’s intelligence operations during the Civil War. 
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time America entered the war, the British, with the participa¬ 
tion of scholars, had developed massive amounts of global 
strategic intelligence. 

It is significant that since the time of Elizabeth scholars 
have played an important role in strategic intelligence produc¬ 
tion. Elizabeth recruited her agents from Cambridge and 
Oxford. Scholars from these same institutions became the 
analysts and researchers who contributed so much to British 
expertise in World War II. In his introduction to J. C. 
Masterman’s The Double-Cross System in the War of 1939 to 
1945 , Norman Holmes Pearson points out that: 

another contribution of British intelligence was that of 
so-called overt intelligence, by which scholarly sources, 
when studied by scholars, revealed information which 
had long since been gathered but already been covered 
by dust. Timetables for tides involved in invasion 
landings, the location of bombing targets within metro¬ 
politan areas, amounts of precipitation to provide 
against, and rainy seasons to avoid: these not-at-all trivial 
data emerged from otherwise ignored pages of books 
rather than from the impossibly delayed reports of 
agents. American intelligence services learned much from 
the British example of research and analysis carried on at 

the universities. 1 ^ 

Likewise in the U.S., General William Donovan, Director of 
the Office of Strategic Services, “ ‘. . . assembled the best 
academic and analytical brains that he could beg, borrow, or 
steal from the universities, laboratories, and museums.’ ” 13 The 
Research and Analysis Branch of OSS, according to R. Harris 
Smith, 


^J. C. Masterman, The Double-Cross System in the War of 1939 to 
1945 (New Haven: Yale University Press, 1972), pp. vii-viii. 

^R. Harris Smith quoting Allen Dulles in OSS: The Secret History of 
America's First Central Intelligence Agency (Berkeley: University of 
California Press, 1972), p. 13. 
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. . resembled a star-studded college faculty. A peek into 
the R & A offices might reveal a heated discussion 
between historian Sherman Kent and political scientist 
hvron Kirkpatrick. A committee meeting of the Eco¬ 
nomics Division might find Charles Hitch, Emile 
Despres, Charles Kindelberger, and Richard Ruggles 
sitting side by side. In other rooms, classicist Norman 0. 

Brown could be writing a report on Greek politics, 
historian John King Fairbank studying an aspect of 
Chinese foreign policy, philosopher Herbert Marcuse 
analyzing German social structure, or anthropologist 
Cora DuBoise pondering the problem of European 
colonialism in Asia. 14 

World War II marks the real beginning of the U.S.’s strategic 
intelligence production capabilities. Although the names of 
intelligence organizations have changed and their missions have 
been modified, major intelligence organizations that exist today 
Bad their progenitors in World War II. 

In lecent years, the introduction of new weaponry has made 
die development of countermeasures imperative. Again a 
change in the manner of resolving conflict has changed the 
nature of strategic intelligence production. Available time to 
react to new threats has been reduced drastically, and this 
reduction in response time makes adequate intelligence (fore¬ 
knowledge) all the more important. The need for adequate 
intelligence has given rise to new intelligence collection devices 
which, m turn, require new technologies to handle the 
voluminous amounts of raw data collected daily. Yet, despite 
sophisticated collection devices and techniques, the man in the 
system is as important as ever. Automated data storage and 
iclneval systems make it possible to manipulate large quantities 
ol data, but only the trained intelligence analyst or researcher 
can make judgments, draw conclusions, or derive meaning from 
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Summary 

U Although the importance of strategic intelligence was 
M’t ognixed and documented as early as 500 B.C., little is known 
about the manner in which this intelligence was produced. 

Strategic intelligence activities by governments became 
apparent in Europe in the sixteenth century. Initially these 
activities were directed toward specific target groups such as 
political dissidents, a specific country, or a religious group. 

Mn the seventeenth and eighteenth centuries, strategic 
intelligence activities by governments began addressing global 
issues in a systematic manner. 

U In World War I, and in the beginning of World War II, U.S. 
intelligence organizations had to rely heavily upon the intelli¬ 
gence produced by the Allied nations. 

U Scholars and academicians were an important part of 
intelligence organizations at least as early as the sixteenth 
century. 

Precursors of all major intelligence organizations were the 
activities carried on by merchants, vendors, and lending houses. 

U Modem technology has revolutionized intelligence data 
collection and processing; however, the man in the system is as 
critical as ever because only he can derive meaning from the 
data. 
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CHAPTER III. 

INACCESSIBLE AND QUESTIONABLE INFORMATION: 
THE NEED FOR INTELLIGENCE 


... A great part of the information obtained in war is 
contradictory, a greater part is false, and by far the 
greatest part somewhat doubtful. 

Karl von Clausewitz 

The winter of 1943-44 was especially severe for the U.S. 
Fifth Army and the British Eighth Army in Italy. The 
northward advance of the Allied forces was halted by a 
stubborn force of Axis defenders at the Gustave Line, a line of 
fortifications built on the commanding terrain overlooking the 
Rapido River and the Liri Valley and anchored on the left flank 
by a monastery on Monte Cassino. This monastery, it was felt, 
was the key defensive position of the entire line. According to 
best opinions at the time, the monastery provided excellent 
cover for German artillery observers who had an unobstructed 
view of Allied forces dug in on the valley floor. Destroying the 
monastery became more than a military objective: it soon 
became an obsession with every Allied commander whose 
casualties mounted daily as a result of enemy action and 
weather. 

Conventional methods failed to destroy what was perceived 
to be the key enemy stronghold. Artillery had little apparent 
effect on the massive stone masonry, and tactical bombing was 
equally ineffectual. The key to destroying the bastion was 
information- information about the construction details of the 
monastery-because only on the basis of this information could 
the optimum size and nature of the destructive devices to be 
used against the monastery be determined. But who had this 
information? Where was it located? How could it be obtained? 
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Exhausting all intelligence staff resources available at 
division and corps, Major General F.I.S. Tuker, Commanding 
(Hl icer of the 4th Indian Division, took matters into his own 
hands. I le drove to Naples, combed book stores until he found an 
old book relating to the monastery, and located precisely the 
Information he needed. On the basis of this information, 
appropriate weaponry was designed, and the monastery was 
destroyed. 1 

l he key to the destruction of this military objective was 
Intelligence -collected, evaluated, and interpreted information 
.icniIicant to military planning and operations. 

Numerous lessons could be derived from the incident 
related above; for example: the necessity of an intelligence staff 
to perform its assigned functions, or the importance of basic 
intelligence in military operational planning. But the feature of 
the incident that is most relevant to this text is the activity that 
General Tuker performed, namely, intelligence research. 

(leneral Tuker perceived an operational situation in terms of a 
problem to solve, delineated his research objectives, considered 
likely sources of information, located the information, 
evaluated it, and made it available to ultimate users. 

This chapter discusses the purpose for which intelligence is 
produced, the conventional forms and components of strategic 
intelligence, and the organizations whose intelligence research 
activities address the various components of strategic intelli¬ 
gence. 

Purpose of Intelligence 

Intelligence is evaluated information. It is produced to help 
policy makers and planners make effective decisions. For the 
greater part, strategic intelligence-that intelligence required for 
use at the national and international level-is used for strategic 
planning. __ 

1 Fred Majdalany, Cassino: Portrait of a Battle (London: Longmans, 
Green and Co., Ltd., 1957), p. 114. Ironically, after the Gustave Line had 
fallen, it was learned that the Germans had not used the monastery as an 
observation post. 
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Strategic planning involves establishing courses of action, 
usually long-range courses of action. A course of action could 
relate to a military activity, but it could also relate to a political 
or economic activity or to a combination of military and 
political activities such as the recognition of a new head of 
state, an embargo, boycott, or blockade. Strategic planning may 
address existing problems or conditions, or anticipated or hypa 
thetical problems and conditions. In the latter cases, the 
strategic planning would be called contingency planning. 

Strategic planning, like all planning, must be based on 
information. However, information required for strategic plan¬ 
ning is often inaccessible because other countries’ military or 
political groups realize the importance that certain types of 
information may have for others who may be real or potential 
enemies. Consequently, these countries take extensive measures 
to deny, limit, or falsify certain types of information and data. 
Thus, in addition to the methodological problems of obtaining 
and utilizing information commonly encountered by most 
planners, strategic planners encounter the very significant 
problem of obtaining information that others wish to deny to 
outsiders. 

This act of denying certain information to others is what 
has given rise to intelligence organizations. Without the denial 
or inaccessibility of certain types of information, intelligence 
organizations would be unnecessary since information required 
for strategic planning would be available from other govern¬ 
mental agencies or from the countries or foreign groups 
directly. 

Not only has the denial of information to others given rise 
to large intelligence organizations and activities, but it has also 
determined the nature of the operations performed by intelli¬ 
gence organizations. For example, intelligence organizations 
must employ an array of data collection procedures and devices 
ranging from sophisticated sensors to human observers. 

In addition to the necessity for using an extensive array of 
collection systems, intelligence organizations must spend much 
time and effort assessing the quality of their data. No estimate 
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i H projection is any more reliable than the input upon which it 
is based; consequently, establishing the reliability of data is one 
nl (lie main functions of any intelligence organization. 

Another characteristic function of intelligence organizations 
I hat evolved because certain types of information are denied is 
the necessity to use inferences when facts are unavailable. Major 
intelligence analyses or research projects are devoted to the 
systematic establishment of strong inferences, to the testing of 
these inferences, and to the gradual substitution of strong 
inferences (and, ideally, facts) for initial tenuous inferences and 
guesses. It is this last characteristic that sets intelligence apart 
Irom many other types of information-gathering activities, and 
i! is the topic that this text addresses throughout. 

F orms and Components of Strategic 
in telligence: Opportunities for Research 

Strategic intelligence encompasses eight major subject 
matter areas and requires the application of knowledge and 
skills of dozens of academic disciplines ranging from informa¬ 
tion science and data processing to anthropology and political 

science. 

There are many ways of classifying intelligence and its 
related activities. However, for the purpose of this text, 
intelligence-the product resulting from the collection, evalua¬ 
tion, analysis, integration, and interpretation of information 
concerning a foreign nation or an area of operations will be 
classified according to two criteria: 1) the basic purpose for 
which it is produced, i.e., its form; and 2) the subject matter 
addressed. These basic forms and subject areas of strategic 
intelligence are described briefly below in order to emphasize 
(lie range of potential types of research that may be required to 
support strategic intelligence production. 

Basic forms of intelligence and the 
purposes for which they are produced 

On the basis of the first criterion, intelligence may be 
produced to enhance the existing body of knowledge about a 
country. This type of intelligence is encyclopedic in nature. It is 
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produced partly in anticipation of kinds of information that 
would be required should a conflict break out involving a given 
country. It is also produced in order to permit better 
interpretations of current activities to be made. This kind of 
intelligence is referred to as basic intelligence. The name is most 
appropriate. Basic intelligence invariably constitutes the bases 
for deriving meaning from other types of intelligence. 

Basic intelligence—its collection and analysis activities—is 
the least spectacular of all types of intelligence operations; but 
nevertheless, this type of intelligence is most essential. It is 
difficult to imagine how the Normandy landings in World War II 
could have been conducted without knowledge of the landing 
beaches- their gradients, load-carrying capabilities, tides, natural 
and manmade obstacles. Basic intelligence is oriented around 
past and present conditions and events. Perhaps the most 
familiar documents containing basic intelligence are the 
National Intelligence Surveys. 

Intelligence is also produced to keep track of events 
occurring in various parts of the world which would impact on 
U.S. policies and national interests. This type of intelligence is 
called current intelligence. It is characterized by change-aptly 
enough- and requires the constant updating of earlier conclu¬ 
sions, analyses, and judgments. When events occur rapidly, this 
type of intelligence gives rise to intense activity. 

Surprises are anathemas to any intelligence organization, 
and one of the fundamental reasons for having intelligence 
organizations is to reduce the likelihood that any foreign power 
could commit an act, develop a capability, or establish a 
-relationship that was unanticipated. Thus, in a further attempt 
to reduce surprise, a third type of intelligence has evolved 
estimative/predictive intelligence. This type of intelligence 
addresses the likelihood of a country’s engaging in a certain 
type of activity in the immediate or forseeable future. Ideally, 
this type of intelligence attempts to establish the intent of 
decision makers in foreign countries. More realistically, estima¬ 
tive/predictive intelligence attempts to anticipate future actions 
based on current capabilities. Although it is concerned with 
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pmgnostication, estimative/prodictive intelligence is founded on 
descriptions of current realities. 

I'valuation is addressed in all three types of intelligence. 
<.)ui(e literally, evaluation is involved in the determination of 
(he actual costs of the other side’s war making machinery, a 
(ask that is normally the purview of basic intelligence. 2 
I valuation might also relate to estimative/predictive functions. 
An example of evaluation used for predictive purposes would be 
determining the enemy’s cost/benefit ratios of alternative 
methods for addressing a scientific or technical problem and 
then predicting the likely course of action on the basis of the 
approach which would provide the greatest payoff for the 
minimum risk. Implicit in this evaluation would be the 
assumption (or the prediction) that the other side would elect 
to follow the more “rational” approach. 3 


2 Appraise would be a more precise term to use since it connotes the 
fixing of monetary worth. 

3 Many predictions in intelligence are based on the assumption that 
the other side is “rational”; and rationality is usually defined in terms of 
the value system of the person making the judgment. This gives rise to 
problems when it fails to consider that value systems may differ 
considerably. The assumption of a “rational” enemy also fails to account 
for behaviors such as Hitler’s “emotional” and fatal decision to bomb 
urban areas during the Battle for Britain rather than to continue to 
eliminate the British early warning systems; or Hitler’s decision to sacrifice 
Paulus’s Sixth Army at Stalingrad rather than yield any conquered 
territory. Surprise in warfare is often possible because it seemingly violates 
the canons of rationality. “Irrational” risks are undertaken when the 
potential payoff is high; hence, the Japanese attack on Pearl Harbor, and 
the Soviet Union’s placement of missiles in Cuba. Given no knowledge to 
(lie contrary, the hypothesis that the “other side” is rational by one’s own 
values is nevertheless a plausible and practical starting point. This will be 
discussed further in later chapters. 
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With respect to tenuousness of conclusions and obsoles¬ 
cence of products, estimative/predictive intelligence ranks 
highest in the potential for mistakes. Estimative/predictive 
intelligence is oriented around present and future conditions, 
activities, and events. 

Components of strategic intelligence 

In addition to being classified by function and time 
orientation, intelligence can also be classified according to 
subject matter addressed. These categories of intelligence 
subject matter are also known as the “components of strategic 
intelligence.” The components include the following: 

Biographic Intelligence 
Economic Intelligence 
Sociological Intelligence 

Transportation and Telecommunications Intelligence 

Military Geographic Intelligence 

Armed Forces Intelligence 

Political Intelligence 

Scientific and Technical Intelligence 

Very briefly, Biographic Intelligence involves collecting and 
evaluating information relating to key personalities in foreign 
countries. Economic Intelligence considers resources and 
capabilities for production, economic vulnerabilities, and the 
availability of strategic commodities. Sociological Intelligence is 
concerned with social stratification, value systems, traditions, 
beliefs, and other social characteristics of selected populations. 
Transportation and Telecommunications Intelligence examines 
transportation networks (highways, railroads, inland waterways, 
civil air) as well as telephone, telegraph, and civil broadcast 
capabilities. Geographic Intelligence studies landforms, weather 
and climate, coasts and landing beaches. Armed Forces Intelli¬ 
gence addresses orders of battle, equipment, doctrine, and 
logistic capabilities. Political Intelligence examines govern¬ 
mental structures, national policy, and political dynamics. 
Scientific and Technical Intelligence is concerned with assessing 
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the scientific and technical capability of a country for achieving 
national goals. 4 

As any student of intelligence will point out quickly, the 
classification of types or categories of intelligence is arbitrary 
and incomplete. For example, Indications and Warning Intelli¬ 
gence is subsumed under current intelligence; and combat, 
tactical, and operational intelligence are excluded entirely in 
this classification. The various “INT’s” of intelligence-SIGINT, 

| I t NT, PHOTINT, HUMINT, and so on-are not addressed 
specifically in this breakdown of types of intelligence, but 
obviously, their outputs contribute to all components and to all 
forms of intelligence. 5 

It should be noted that each component of strategic 
intelligence is concerned with each of the three forms of 
intelligence. In Biographic Intelligence, for example, an indivi¬ 
dual who shows promise of rising to positions of authority 
would be a subject pertinent to estimative/predictive intelli¬ 
gence analysis and research. Individuals currently in power or 
currently occupying leadership positions would be addressed by 
current intelligence analysts. And individuals who formerly held 
high positions but who were replaced, abdicated, resigned, or 
tlied, become the subjects of basic intelligence research or 
analysis. In short, even “static” subject matter becomes 
seemingly “dynamic” in strategic intelligence production. 


4 For a more detailed description of the components of strategic 
intelligence, the reader should consult Introduction to the Components of 
Strategic Intelligence (Washington, D.C.: Defense Intelligence School, 
1972). 

5 To confuse matters further, some intelligence organizations are 
organized on a geographical basis, e.g., China/Asia Division, Western Area 
Division; or on an analytical function versus an operational function basis; 
or on a foreign intelligence, domestic counter-intelligence basis, and so on. 
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Within the total framework of forms and components of 
strategic intelligence, two basic functions of intelligence are 
eyident: 1) to explain, account for, or describe a phenomenon; 
and 2) to predict, forecast, or estimate. 6 These two functions 
will be recurring themes throughout the rest of this text. 

Research Within the 
Intelligence Community 

Clearly, the scope of strategic intelligence collection and 
production exceeds the capabilities of any one organization. 
Furthermore, not all government organizations are equally 
concerned with, nor are they the primary consumers of, all 
categories of intelligence. It is not surprising, therefore, that’the 
State Department would be more concerned about Political and 
Sociological Intelligence than would the Department of De¬ 
fense. Conversely, intelligence directly relevant to military 
operations—Armed Forces Intelligence, Military Geographic 
Intelligence, and Transportation and Telecommunications 
Intelligence—would be of greater concern to the Department of 
Defense and, more specifically, to the Defense Intelligence 
Agency (DIA) and to the intelligence organizations of the 
various services, than to either the Central Intelligence Agency 
(CIA) or to the State Department. Scientific and Technical 


Estimation addresses two types of problems. For example, an analyst 
might be assigned the task of estimating the size of an enemy garrison. In 
this case the enemy garrison comprises a finite number of personnel at any 
specific time which, theoretically, could be counted. On the other hand, 
an agricultural economist might be assigned the task of estimating the size 
of the Soviet Union’s wheat crop-a crop that would not be ready for 
harvest for months to come. This type of estimation involves more than 
counting. It requires the analysis of those variables that would affect the 
crop (precipitation, temperature), and the determination of the effects of 
these variables; hence, it involves prognostication. The first type of 
estimation is essentially description. The second type of estimation 
involves an element of prediction. 
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Intelligence would be of special interest to DIA and CIA 
because of the potential threat implications of new tactical and 
strategic weaponry, but this type of intelligence would not be a 
major concern of the State Department. 

Production responsibilities of the various components of 
strategic intelligence are shown below. Table III-1 denotes the 
primary responsibility of an intelligence community member: it 
does not imply that other members of the community may not 
produce that type of intelligence. In instances in which current 
intelligence requirements dictate the necessity of one agency’s 
producing intelligence which would nominally be the respon¬ 
sibility of another agency, permissive responsibility for produc¬ 
tion may be granted to that agency. 

The intent of this brief description is not to teach the 
organization of the U.S. intelligence community, but rather to 
indicate in which agency certain types of intelligence research 
functions would most likely be performed. The agency responsi¬ 
ble for producing a certain kind of intelligence can be expected 
(o have the greatest amount of reference materials, sources and 
resources relating to that particular topic area. 

But there is a danger in thinking that certain types of 
intelligence are necessarily the purview of one organization and 
not another. For example, in stability-type operations, the U.S. 
Army would be both a producer and consumer of Biographic 
Intelligence, Political, and Sociological Intelligence. And in 
psychological operations (PSYOP), for example, Sociological 
Intelligence might be produced and consumed by joint State 
Department and U.S. Army teams. An example of the latter was 
the Joint U.S. Public Affairs Office (JUSPAO) in Saigon which, 
along with MACJ3-11 of the U.S. Military Assistance Command 
in Vietnam (MACV), coordinated and conducted PSYOP 
activities throughout Vietnam. JUSPAO and MACJ3-11 utilized 
Biographic Intelligence, Political, Sociological and Telecom¬ 
munications Intelligence in its day-to-day operations. 

In World War II, the paper “Tenth Fleet” of the U.S. Navy 
produced and utilized Political and Sociological Intelligence on 
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Table 111-1. Intelligence Organizations and their 
Primary Production Responsibility 


COMPONENT OF STRATEGIC 
INTELLIGENCE 

PRIMARY RESPONSIBLE AGENCY 

BIOGRAPHIC 

DIA (MILITARY)* 

CIA (OTHERS) 

ECONOMIC 

STATE DEPARTMENT (FREE WORLD) 

CIA (COMMUNIST COUNTRIES) 

SOCIOLOGICAL 

STATE 

TRANSPORTATION/TELLECOMMUNICA¬ 
TIONS 

DIA 

MILITARY GEOGRAPHIC 

DIA 

ARMED FORCES 

DIA 

POLITICAL 

STATE 

SCIENTIFIC/TECHNICAL 

DIA (AND VAR 1 US S&T ORGANIZA¬ 
TIONS OF THE DIFFERENT 
SERVICES) 

CIA 

•AS OF OCTOBER 1974 THE NATIONAL SECURITY AGENCY (NSA) WOULD 
ADDRESS ALL COMPONENTS AS OPPORTUNITIES EXISTED 
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internal conditions in Germany as well as current tactical 
intelligence on German U-boat dispositions. And U.S. Navy 
personnel, notably Rear Admiral Ellis M. Zacharias, used his 
knowledge of social customs, traditions, and key personalities 
of Japan effectively in the conduct of propaganda campaigns 
Irom the Office of War Information (OWI). Again, the situation 
and policy dictate the manner in which a single agency or a 
combination of agencies will address an intelligence problem. 

Summary 

l> Intelligence, collected and evaluated information about a 
foreign country or an area of operations, is essential for 
strategic planning. 

l> That many countries deny certain information to outsiders 
makes intelligence organizations and activities necessary. 

O The denial of information determines both the nature of the 
activities conducted by intelligence organizations, and the 
nature of the products these organizations produce. 

t> Strategic Intelligence can be classified on the basis of the 
various forms of intelligence as well as by the subject matter 
addressed in the various forms. 

- Forms of intelligence include basic intelligence, current 
intelligence, and estimative/predictive intelligence. 

- Components of strategic intelligence, i.e., the subject 
matter areas addressed, include Biographic, Economic, 
Sociological, Transportation and Telecommunications, 
Military Geographic, Armed Forces, Political, and 
Scientific and Technical Intelligence. 

t> Various agencies of the intelligence community have dif¬ 
ferent primary responsibilities for intelligence production. 
These agencies are also the primary repositories of information 
relating to their areas of responsibility. 
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CHAPTER IV. 

RESEARCH: A DESCRIPTION OF THE ACTIVITY 


. . . The idea that the object of research is new know¬ 
ledge does not seem to be widely understood and *a 
schoolboy looking up the meaning of a word in the 
dictionary is now said to be doing"research.* Indeed , it 
has been debased even further. Research is frequently 
used to describe reading by those to whom reading , 
apparently, is a recherche activity, and for many a 
graduate student it is a euphemism for wholesale 
plagiarism The work needs a rest or at least less 
promiscuous handling. 

Bergen Evans and Cornelia Evans 

Many activities referred to as “research” do not satisfy the 
requirements of the term. This chapter describes briefly the 
various interpretations of the term, and sets forth three criteria 
that must be satisfied in order for an activity to qualify as 
research. 

Original Meaning and 

Modern Permutations 

“Research” (or re-search) evolved from the French verb 
recherche which, strictly speaking, means to search again, or to 
explore something that has already been explored. The activities 
of Copernicus and Tycho Brahe would qualify well as examples 
of research in the original sense of the word. Both astronomers 
examined their recorded observations of the movements of 
heavenly bodies time and time again. Their contributions to the 
science of astronomy were significant not so much for the 
addition of new facts but for their better interpretations of the 
then-existing information. 

Today, the generally accepted meaning of the word “re¬ 
search” is diligent and systematic inquiry into a subject to 
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discover new information or principles. Admittedly, if the 
lormulation of principles were the only criterion, very few 
"diligent and systematic inquiries” would qualify as research. 
More commonly, research involves the formulation and testing 
ol hypotheses, those precursors of the more elusive theories, 
principles, or natural laws. 

From the standpoint of semantics, the word “research” 
should be defined rigorously and its use restricted or it should 
he stricken from the English vocabulary. Denotatively, the 
meaning of the original term is quite explicit: namely, to search 
again, to explore something that has already been explored. 

(bnnotatively, however, “research” means much more. A 
cursory examination of the titles of theses submitted by 
candidates for advanced degrees reveals that activities involving 
experimentation, surveys, examination of historical documents, 
development of techniques and algorithms, unusual applications 
of established methodologies, testing of materials, and the 
creation of artistic works, have all qualified as “research”- 
qualified in the sense of satisfying the requirements for an 
advanced degree. 

On a typical university campus one can expect to find as 
many different interpretations of what constitutes research as 
there are academic disciplines represented in the sample of 
respondents. And the person performing research in intelligence 
is very likely to encounter researchers to whom any investiga¬ 
tion outside of their own particular specialities would not 
warrant the title of research. Hence, the physical scientist (and 
many behavioral scientists) would perceive research as an 
activity involving some sort of an experiment, a historian might 
perceive research as an activity involving an intensive examina¬ 
tion of records of events; an analytical philosopher might 
perceive research in terms of a critical examination of the 
meaning of terminology used in philosophic discourse; and an 
engineer could perceive research in terms of submitting various 
materials or structures to tests. 

Not only is the term “research” used to encompass a wide 
range of activities pertaining to an even wider range of subject 
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matter areas, but it is sometimes also used to add a degree of 
respectability to activities that otherwise would be considered 
trite and mundane. 

Numerous attempts have been made to dictate the desired 
use of terminology, but generally these attempts fail. The 
Queen of Heart’s disclaimer to Alice that “words mean 
whatever I want them to mean” is an apt description of most 
users of modern languages. 

But meanings are obscured, concepts cannot be shared, and 
misunderstandings result with the use of words that mean 
different things to different people. Therefore for those investi¬ 
gators or students who will be faced with the requirement to 
perform research, an understanding of the term is most 
important. From a very practical standpoint, an understanding 
of the more precise meaning of the term will help eliminate 
time lost in planning and conducting an activity which would 
not qualify as research. This understanding will also broaden 
one’s perspective of the wide range of activities (and products) 
that do qualify as research. 

For example, the authors recall an incident in which a paper 
that traced the creation and evolution of a military unit was 
shown to a psychologist. The psychologist read the paper and 
commented that the paper qualified as a “piece of journalism” 
but hardly as an example of “research.” The report, which 
corrected a number of misstatements that had been promul¬ 
gated for years, was compiled after a lengthy and painstaking 
examination of hundreds of documents contained in reposi¬ 
tories in three countries. The psychologist’s comment is 
pertinent for two reasons: first, it revealed a bias that any type 
of inquiry not cast in a certain framework—in this case an 
experiment, the research model with which the psychologist 
was most familiar—was not acceptable as legitimate research. 
Second, by implication, the comment denigrated journalistic or 
reportorial research, an activity which when done well involves 
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extensive “systematic inquiry” and requires the same amount of 
attention to detail, concern about sources, objectivity, and 
Ihoroughness that should characterize all research. 

It should be apparent that an activity does not qualify as 
research merely because certain subject matter is addressed or 
because certain methodologies are employed. However, an 
activity may qualify as research provided three criteria are met: 

1) the activity is purposeful , i.e., it is conducted with a specific 
objective in mind; 2) the activity is performed systematically; 
and 3) the activity contributes new knowledge (or new 
interpretations of old knowledge). These criteria warrant 
further discussion. 

Researc h as a Purposeful Activity 

In research, purposefulness implies the existence of an 
objective-a clearly identifiable, specifiable, and meaningful end 
in its own right which, if achieved, could be recognized not on y 
by the researcher, but also by independent observers as well. 
For example, an intelligence analyst who constantly updates an 
order of battle (OB) is not performing research by this criterion 
since orders of battle always require updating. The updated 
order of battle is tentative and is only a means to some other 
end. A technician performing quality control tests on materia s 
is not performing research because, again, no specific identifi¬ 
able end is addressed. Testing is merely a set of procedures for 
maintaining a level of quality; it is not an end in itself. 

On the other hand, an OB specialist would be performing 
research if he formulated hypotheses relating to a command 
hierarchy or the location of an unnamed unit and systemati¬ 
cally tapped his sources to confirm or refute his hypotheses. 
And a quality control technician could be said to be performing 
research if he devised and tested a procedure for sampling. In 


1 Significantly, journalism is an excellent model for intelligence 
researchers. The time constraints under which reporters typically operate 
and the consequences of publishing incorrect, or incomplete information 
are most analogous to the intelligence researcher’s constraints. 
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both examples, the criterion of addressing a specific, finite 
objective was satisfied and a clearly identifiable end product 
resulted. 

Aims, objectives, and goals are three terms commonly 
associated with purpose. As they are used in this text, aims refer 
to the general intents of the investigator. Aims are manifested in 
the direction that an investigator’s endeavors might take 
Objectives, on the other hand, refer to clearly specifiable and 
identifiable ends of an investigation. Objectives are often stated 
in terms of some finite time period. Goals refer to those 
ultimate ends which, for most practical purposes, are never 
achieved. Thus, the aim of military planners might be to sever 
the enemy’s lines of communication. The aim could be cast in 
terms of specific objectives, e.g., destroying a railroad 
marshalling yard, a bridge, or a critical stretch of highway 
within a specified period of time. The ultimate goal of these 
endeavors might be to bring about a lasting peace,” or “to make 
the world safe for democracy.” 

In the context of intelligence research, the aim of the 
researcher might be to study North Korean propaganda and 
North Korean political or military behavior. His specific 
objective might be to determine whether or not any correlation 
existed between North Korean public announcements relating 
to unification and incidents along the DMZ during the period 1 
January 1972 to 4 July 1972. The overall goal of the researcher 
might be to formulate generalizations about the words and 
deeds of North Korean policy makers that might be used as a 
basis for anticipating future behavior. 

Research as a Systematic Ac tivity 

In order to qualify as research, the act of inquiry must be 
conducted in a systematic manner. The exact nature of the 
system employed, of course, would be a function of the subject 
matter addressed. Systematic implies that an investigation 
adheres to an organized and methodical order of procedures-a 
plan, for example. The main reason for emphasizing a systema¬ 
tic and orderly approach to an investigation is to ensure that 
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significant elements are not excluded. For example, there are 
certain critical steps in experiments which, if they were not 
addressed initially, would render later findings worthless. 

Another reason for emphasizing a systematic approach is to 
make explicit the manner in which conclusions were reached. 
Systematic procedures make it possible for other researchers to 
replicate a study in order to determine if the findings 
necessarily derived from the data and not from some spurious 
element accidentally (or intentionally) introduced by the 
investigator. 

One of the most devastating things that can befall a 
graduate student is to be informed at the time he is defending 
his thesis that he overlooked a significant work. Equally 
frightening, and potentially far more serious for the intelligence 
researcher, is to be in a situation of defending an estimate that 
overlooked significant data held by another agency. Attention 
to detail and an organized and orderly manner of progression 
throughout the research phases are ways of precluding these 
nightmares. 

Research as an Activity that 
Contributes New Knowledge 

The final criterion of research is that it contributes new 
knowledge. Ultimately, this is the most significant criterion and 
the end toward which the other two criteria contribute. 

At the outset it should be made clear that new knowledge is 
not necessarily synonomous with new information. For 
example, new interpretations of old information would qualify 
as new knowledge in those instances in which the investigator 
could make a plausible case for his new assertions. (Copernicus, 
for example, in disproving the geocentric theory in favor of the 
heliocentric theory did precisely that.) Much historical research 
utilizes essentially no new information but instead re-examines 
existing information in order to formulate and test new 
hypotheses. Obviously, obtaining new information for 
generating new knowledge plays a significant part in many of 
today’s research programs, and it is highly likely that in most 


35 













intelligence research, new information or data would have to be 
obtained. 

Satisfying this criterion of research creates its own array of 
nightmares for the researcher. Every graduate student lives in 
tear that, unbeknownst to him, some researcher at some distant 
university is addressing the very same problem as he and that 
the other researcher might publish his findings first. Scientists 
on the verge of major breakthroughs are highly competitive in 
their desire to publish before their colleagues, as James D. 
Watson candidly admits, 2 and rivalry between intelligence 
organizations of the same country is by no means unusual, as 

C ookndge points out in his description of the Gehlen organiza¬ 
tion. J b 

Summary 

> In order to qualify as research, an activity of inquiry must 
satisfy three criteria. 

I> The activity must be purposeful in the sense that it 
addresses an identifiable end or objective; e.g., that it attempts 
to answer a specific question or set of questions. 

t> The activity must be performed systematically: it must 
conform to a plan or a design. 

> The activity must contribute new knowledge 0 r new 
interpretations of existing knowledge. 


-James D. Watson, Double Helix: Being an Account of the Discovery 
<>) the Structure ofDNA (New York: Atheneum Publishers, 1968). 

!| ' H ' Cookridge, Gehlen: Spy of the Century (New York: Random 
House, 1972). 


CHAPTER V. 

INTELLIGENCE RESEARCH/ACADEMIC 
RESEARCH: SIMILARITIES 
AND DIFFERENCES 


Many of those now engaged in intelligence production 
have formerly been engaged in creative scholarship at 
educational institutions. They find much similarity. 

Those who have a sincere interest in their work, high 
standards of scholarship, and a proper pride in the 
finished paper find that those values are also of great 
importance in intelligence work. Many intelligence 
workers, therefore, quite naturally overlook the funda¬ 
mental difference between an intelligence paper and a 
scholarly paper. 

Washington Platt 

The layman typically associates research with universities. 
But obviously research is conducted in many non-university 
settings as well, for example, in industry, in private institutions, 
and of course, in numerous governmental agencies. Despite the 
varieties of activities performed under the rubric of research, 
despite the subject matter addressed, and despite the organiza¬ 
tion sponsoring the activity, the basic concept of research that 
persists is the academic model—academic in the sense that the 
activity conforms to certain scholastic traditions or rules. 
Academic research must be purposeful, systematic, and con¬ 
tribute new knowledge. But in addition, the activity or the 
product should exemplify reasoning ability, judgment, intel¬ 
lectual honesty, and objectivity on the part of the individual 
performing the activities. 

Although the canons of academic research still constitute 
the underlying model for intelligence research, often the canons 
cannot be satisfied. Thus, in many respects, intelligence research 
may be the antithesis of academic research. This chapter tells 
why. 
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Time Constraints 


With the possible exception of journalists, no professionals 
work under the constant press of time constraints so much as 
intelligence analysts or researchers. Admittedly, the graduate 
student anticipating a June graduation works under a kind of 
time constraint, and a researcher-Watson, for example-who 
wants to cop a Nobel Prize works under a competitive time 
constraint; but these constraints are artificial in the sense that 
survival is seldom at stake. With respect to urgency, perhaps the 
closest analogy to intelligence research would be medical 
research directed toward developing immunization for a parti¬ 
cularly virulent disease that was decimating a population. 

Time constraints ultimately affect other characteristics of 
intelligence research as well; for example, the intelligence 
researcher may not have enough time to collect all of the 
information that he felt was essential to arrive at a conclusion, 
or he may not have the time to corroborate or establish the 
validity of his input and, consequently, must qualify his 
conclusions. It should be noted that the individual with a 
penchant for an unhurried and exhaustive analysis of his 
research problem may find that he is exceedingly unhappy and 
frustrated by the necessity to produce estimates long before he 
is satisfied with his input or with his analysis. As the nature of 
conflict changes, the intelligence researcher may find that the 
tempo of his activities changes accordingly. Time constraints 
may induce an intolerable level of stress. Admittedly, some 
individuals not only adapt to the necessity of producing under 
stress, but also seem to thrive in this kind of environment. 
Other individuals whose perception of research was limited to 
the academic model may never learn to adapt and eventually 
drop out of (or are dropped from) the profession. 

Control of Variables 

Much academic research involves experimentation. An 
experiment is essentially a set of procedures used to test an 
hypothesis. The experimental ideal is a situation in which the 
independent variables can be manipulated and controlled in 
such a manner that the dependent variable follows 
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unambiguously and irrefutably. The outcome of the experiment 
is determined by the design and by the controls imposed by the 
design. These conditions can be created and controlled fairly 
well in the laboratory: in the field, however, experimental 
controls are rarely possible. Many social scientists (and 
inexperienced intelligence researchers) are dismayed when they 
discover that the experimental ideal they addressed so 
effectively in the university setting is so elusive in the field that 
it tends to be nearly irrelevant in application. Again, many 
researchers learn to adapt, to be pragmatic, to alter and modify 
their approaches to accommodate the constraints imposed on 
(heir research activities. Other researchers feel that adaptation is 
tantamount to compromising standards and consequently refuse 
to work in this type of environment regardless of the fact that 
decisions must still be made and the decisions will be made with 
or without the knowledge that even a “compromised” 
experimental design might yield. 1 

In intelligence research, the researcher has virtually no 
control of the variables other than the methods he employs 
himself. The enemy, or the “other side,”is notably uncoopera¬ 
tive. The enemy may refuse to respond to a “stimulus” (for 
example, a propaganda message), he may deliberately alter his 
mode of behavior in a manner to avoid any indication of 
pattern, he may change his response modes constantly in order 
to preclude anyone’s anticipating a future action. In short, and 
in the language of the social scientist, the other side is a 
notoriously poor subject; yet he is the subject of inquiry to the 
intelligence researcher. 


lFor a more complete discussion of the constraints placed on social 
research in a field situation see Ernest F. Bairdain and Edith M. Bairdain, 
Final Technical Report, Psychological Operations Studies- Vietnam, vol. 1 
(McLean, Virginia: Human Sciences Research Inc., May, 1971). 
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Adequacy of Data Versus the 
Necessity to Report 


Incompleteness is a feature that characterizes nearly every 
intelligence product. Sometimes an assessment or a prediction is 
incomplete because no data exist. Data may not exist because 
the other side may not have manifested any observables. 2 More 
often, an intelligence product is incomplete because the enemy 
does a good job of denying certain kinds of information to 
outsiders. It is this type of situation that the intelligence 
researcher is most likely to encounter. In both instances, 
however, the intelligence researcher is required to arrive at 
conclusions—conclusions which may have profound implica¬ 
tions for national security—but conclusions which are founded 
on theories, assumptions, and sometimes simply guesses when 
data or information are not available. 

Unlike academic research in which the researcher may delay 
publishing his findings until he or his advisers are satisfied that 
the findings are adequate to justify his conclusions, the 
intelligence researcher may not have time to collect additional 
data, or the political or military situation may have changed so 
much that additional data could not be collected even if time 
were available. Again, this condition induces much frustration 
in those to whom the academic model of research is inviolable. 

Unknown Quality of Data 

The academic researcher attempts to use data of the highest 
quality. He checks sources, corroborates information, excludes 
information from notably unreliable sources, and withholds 
publication until he is satisfied that his conclusions are founded 
upon irrefutable evidence. The intelligence researcher attempts 
to emulate his academic counterpart but often encounters 
situations in which the only data available are data from sources 
whose reliability has not been established. Furthermore, the 
intelligence researcher sometimes encounters data that have 


2 To put it another way, the other side may take a “wait and see” 
stance and, thus, not react to a situation in any overt manner. 
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been prepared deliberately to deceive or to mislead. 
Admittedly, the academic researcher encounters these situations 
at times, for example, the spurious “Donation of Constantine 
and the “Piltdown Man,” but these exceptions are so rare as to 
be noteworthy perversions in their own right. In certain types 
of intelligence research activities, on the other hand, these 
noteworthy exceptions tend to be the common rule. Again, the 
intelligence researcher is required to publish despite his lack of 
confidence in the data simply because in many instances 
operational decisions cannot be deferred until more or better 
information is available. 

Emphasis on Prediction 

Of all the intellectual activities performed by man, none is 
so fraught with potential for error as prediction. Yet prediction 
(in its various forms) constitutes a major part of the activities in 
intelligence, and is, in fact, the purpose for which even basic 
intelligence is produced (fore-knowledge). Prediction, for the 
greater part, is based upon observed regularities. For example, a 
certain combination of meteorological phenomena invariably 
results in another predictable type of phenomenon. Natural 
laws are essentially predictive statements which take into 
account the interaction of regularly occurring phenomena. 

The problem of prediction in intelligence is exacerbated by 
two conditions: first, unlike most naturally occumng 
phenomena, humans have the ability to alter and control certain 
conditions and interactions. A cautious enemy, for example, 
will avoid repetitive acts which would permit his opponent to 
anticipate his future behavior; or the enemy may deliberately 


3gee, for example, Ladislav Bittman, The Deception Game: 
Czechoslovak Intelligence in Soviet Political Warfare (Syracuse, New York: 
Syracuse University Research Corporation, 1972). 


interestingly, the science of astronomy evolved from the necessity of 
priests to be able to predict movements of heavenly bodies. The ability to 
predict enabled the priests to exercise power, and today the relationship o 
predictive ability and power still holds. 
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maintain one pattern in order to conceal another type of 
as 5 6 * f^ 10 h r ' As SU( 7 the Y^eHigence researcher may be uncertain 
behavior° W mUCh Credlbl!lty he can assi g n to an observed 

nrf> P. e second condition that gives rise to difficulties in 
prediction are unique events-events which have no precedent 
Examples of such events might be the appearance of a new RF 
signal in an area in which all signals had been accounted for- or 
e enemy s construction of a peculiar type of structure whose 
purpose cannot be determined from its shape or size. Although 
procedures exist for dealing with unique events, such as the 

nonTof th ° f aIt r ati I e SC , enarios and g am ing, for example, 
none of the procedures has the predictive power for a researcher 

to state unequivocally that, given conditions “X” and “Y ” “7” 

will result invariably. Yet this is the kind of assertion that 

policy makers seek from intelligence organizations. Much has 

been said about the weighty responsibility of the decision 

mat^ th Ut l e attentlon has been paid to the person who 
makes the predictions upon which decisions are made How 

many people for example, recall the name of the meteorologist 
hivasion? 6 ^ 10 ^ C 6anng weather P rior to the Normandy 

Lest the picture be painted too bleakly for intelligence 
research, it should be noted that an increasing amount of 
academic research is also addressing predictive-type prob- 
ems, for example: predicting volumes of vehicular traffic in 
urban planning, predicting mineral yields, predicting the effects 

nr^^°? U atl ° n ° n ground water Quality and consumption, 
predicting economic impact of new industry in a region, and so 
on. In addition, courses on “futurology” are becoming in¬ 
creasingly common on campuses throughout the country. 

Futurdogy, as the name implies, is the study of future 
conditions based on current and anticipated trends. In 1974 


5 RAf Group Captain J. N. Stagg gave the final meteorological report 

and prediction to General Eisenhower and his staff prior to D-Day. 
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some 200 courses in various related areas of “futurology” were 
offered at 140 institutions, compared to a single course offering 
in 1966. 6 

Sources of Information 

Anyone who is even remotely familiar with national 
intelligence organizations realizes that these organizations have 
means of obtaining information that are unavailable to the 
general public. The ability to obtain certain kinds of informa¬ 
tion by use of the more esoteric collection devices have made 
obsolete some of the traditional methods of research. For 
example, in certain instances in the past, analysts and re¬ 
searchers had to guess the number of enemy surface units or 
aircraft whereas today in many cases they have the capability of 
actually counting them. The ability of the intelligence re¬ 
searcher to draw upon the information yielded by these more 
esoteric collection devices gives him a clear-cut advantage over 
the researcher who is constrained to use more conventional 
methods of data collection. Perhaps the biggest change that has 
resulted from the use of newer collection devices is the 
confidence that can be placed in the results of the analyses. 
Research procedures, per se, may remain essentially unchanged, 
but given “harder” data, the researcher is in a much stronger 
position to draw conclusions. 

The intelligence researcher also has access to inter-agency 
libraries and reference services that rarely would be available to 
the student or academician. In terms of the size of the U.S. 
Government’s holdings to which he has access and in terms of 
the automated retrieval systems at his disposal, the intelligence 
researcher is in a far more advantageous position to conduct 
research than his academic counterpart. 

Emphasis on Security 

To a real or potential enemy, the awareness that the other 
side is attending to certain conditions or events within his 


^Kathy Nagurny, “When is Man Going to Learn from the Future?”, 
The Philadelphia Inquirer, 1 December 1974. 


43 














country provides a tactical advantage. Consequently, intelli¬ 
gence organizations avoid advertising what they are examining 
and how they are going about doing it. This security is essential 
to safeguard human and other sources of information as well as 
to avoid embarrassment at diplomatic levels. To the analyst or 
researcher, the result of this emphasis on security is anonymity. 
This usually means that the analyst may not discuss his 
activities with his academic counterparts, and that the product 
of his efforts will not be attributed to him personally. 7 Many 
researchers whose psychic rewards come in the form of papers 
published in professional journals find that the enforced 
anonymity is intolerable. Other researchers, of course, learn to 
accept this condition. 

Again, the picture is not all bleak for those who desire to 
perform both academic and intelligence research because there 
are many types of intelligence research activities that need not 
be classified and probably will not be classified in the future. 

Secrecy in intelligence will always exist. But changes are 
occurring rapidly, and activities that were formerly classified (or 
not acknowledged) are now openly admitted. It would take an 
extremely naive individual, for example, to assume that the 
Soviet Union or the People’s Republic of China were not 
primary concerns of U.S. intelligence organizations. 

Utility 

To the question, “Why do you climb mountains?” George 
Leigh-Mallory responded, “Because it is there.” Poetic perhaps, 
and provocative, this response would not suffice should an 
intelligence researcher be asked why he was “researching” a 
certain topic. Although all research is purposeful in the sense 
that it attempts to provide new knowledge, intelligence research 
must address a utility goal as well. In other words, it must assist 
in solving a problem—most likely, a decision-making or policy 
planning problem. 


7 The re may be changes to this policy. DIA is considering the use of 
by-lines on certain reports. 


44 


Because intelligence research serves utilitarian purposes does 
not mean that “scholarly” research has no place in intelligence. 
The criterion of utility says nothing about the subject matter o 
the research nor anything about the research methodology, the 
utility criterion simply means that the product of the research 
must address either an existing need or a plausible potential 
need: in short, intelligence research must yield more than 
“truth for truth’s sake.” 

At times the precise manner in which a research product 
will be (or could be) used may be difficult to predict. For 
example in 1965, Dr. William Stockton (an anthropologist and 
a former student at the Defense Intelligence School), along with 
three other investigators, prepared a report dealing with 
indigenous tribes of the Republic of Vietnam. 8 Although the 
overall relevancy of such a work was apparent even in 1965 
little did the authors realize that the sections of the report 
relating to burial habits would be extremely useful nine years 
later when attempts would be made to ascertain the fate of 
downed U.S. airmen. Interestingly, Dr. Stockton also partici¬ 
pated in this latter activity as well. 

A seemingly “academic” research project such as deter¬ 
mining precedents (if any) of the Russians’ use of massed 
artillery (as in World War II) may well provide a basis for 
anticipating Soviet artillery doctrine in future conflicts And an 
exhaustive examination of France’s employment of static 
defenses, such as the Magmot Line and the fortresses of Dien 
Bien Phu, might reveal subtleties in French military doctrine 
that would not be apparent at a superficial level. Current 

8 Joann L. Schrock et al, Minority Groups in the Republic of Vietnam 
(Washington, D.C.: Cultural Information Analysis Center, Center for 
Research in Social Systems of the American University, 1966.) 

ironically, the fall of Dien Bien Phu and the subsequent decline of 
French influence in Indo-China was brought about largely by the Viet 
Minh’s employment of massed artillery. The French defenders of Dien 
Bien Phu ruled out the likelihood that the Viet Mmh would or could use 
artillery because of the mountainous terrain and the absence of roads, a 
classic example of an intelligence failure. 
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conflicts between the Soviet Union and the People’s Republic 
of China had precedents as early as the thirteenth century when 
the Mongol hordes sacked Russian settlements in the Donetz 
basin. An analysis of these early conflicts may throw some light 
on the present Sino-Soviet conflict. 

Research topics evolve from perceived needs of the opera¬ 
tional situation, and many times topics for investigation will be 
assigned to the researcher rather than chosen by him, as will be 
discussed later. But the intelligence researcher who is 
thoroughly familiar with a body of subject matter is in an 
excellent position to anticipate research requirements, and in 
these instances the researcher can often define and select his 
own research topics. 

Briefly then, because a research topic does not have any 
immediate utility should not necessarily discourage an intelli¬ 
gence researcher from pursuing it. It is the potential usefulness 
as well as the immediate usefulness of the product that 
establish its worthiness as a suitable topic for research in 
intelligence, as evidenced by the large amounts of basic 
information contained in the old National Intelligence 
Estimates. 

Summary 

D> Intelligence research attempts to emulate the academic 
model of research in the sense that it is (or should be) 
purposeful, systematic, and it provides new knowledge. 

t> Unlike academic research, intelligence research is con¬ 
strained by limited time, little or no control over experimental 
variables, insufficient data, and the necessity to use data of 
unknown quality at times. 

> Apart from constraints, intelligence research differs from 
academic research in four other respects as well: 

Much emphasis is placed upon prediction in intelligence 
research, whereas description and interpretation pre¬ 
dominate in academic research. 


Qr _ h uses means of collecting data that 
' SSFW Cpatt£fo?“„y university or private 

_ Sle'academic research findings, in.eliigence research 

findings are usu^ly classified- must adclr ess an 

- «—**■ more ,han 
truth for truth’s sake. 

> Because the academic j-esearch^oeTnot imply 

approximated at tim ® s y l " less 8 rigorous or degraded form of 
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CHAPTER VI. 

CHARACTERISTICS OF THE INTELLIGENCE RESEARCHER 


Not only are we uncertain about the strength of the 
enemy , but in addition rumor ... exaggerates his size. 

The majority of people are timid by nature , and that is 
why they constantly exaggerate danger. All influences 
. .. combine to give... a false impression of [ the] 
opponent's strength , and from this arises a new source of 
indecision. 

Karl von Clausewitz 


Ask any authority to identify those human traits that are 
absolutely essential for anyone proposing to carry out research 
and four traits are mentioned invariably: reasoning ability, 
accuracy, intellectual honesty, and open-mindedness. Other 
traits that are mentioned very often include judgment, 
diligence, thoroughness, originality, and so on. There is no 
question that the possession of these traits is a necessary 
condition for producing an acceptable research product: but 
the possession of these traits alone is not & sufficient condition 
to guarantee that a research product will be acceptable, 
f urthermore, because of the uniqueness of intelligence research, 
certain additional human traits are required as well. Finally, 
even the more familiar traits have special implications for the 
intelligence researcher. These necessary (but not sufficient) 
traits of the intelligence researcher are discussed below. 

Reasoning Ability 

Invariably, any individual thinks he reasons well and that 
the inability of others to accept his conclusions are reflections 
of their inadequacies, not his. Little, Wilson, and Moore point 
out that: 


it is not easy to see the errors in our own thinking. Many 
of us are confident that our thinking is above error. But 
confidence is no guarantee. In fact, those who seem 
most confident in their thinking are usually the ones 
who fall into the most serious errors. Fanatics are far 
more confident in their conclusions than are normal 
people .... We may believe we have thought matters 
over thoroughly when, as a matter of fact, we have 
merely unconsciously accepted beliefs or conclusions 
from someone else and then done a bit of window 
dressing to convince ourselves and others that the 
conclusion resulted from logical procedures. 1 

Lxamples of the inability to reason well abound. It is not 
uncommon to find analysts failing to distinguish between facts 
and inferences or operating on the assumption that an 
inference was a fact. It is not unusual to hear an analyst 
announce that his conclusions followed “logically” from the 
evidence, even though generalizations arrived at inductively are 
not subject to logical proof. That different types of inquiry are 
subject to different types of “proof’ is an alien ^concept to 
many researchers. And the common misuse of “infer” and 
“imply” reflects not only a lack of knowledge of terminology 
but also an unfamiliarity with underlying concepts of logic as 
well. 

The seemingly simplistic solution of requiring the researcher 
to “take a course in logic” will not necessarily guarantee an 
improvement in his reasoning ability. But an exposure to logical 
fallacies, a familiarization with deductive and inductive pro¬ 
cesses, and an exposure to those subjective factors which 
influence thinking, should at least heighten the researchers 
sensitivity to the potential for error. An exposure to logical 
fallacies, for example, may reveal to the researcher fallacies in 
his own thinking of which he was unaware. The researcher will 
also discover that, despite the vigor of approach, conclusions 


'Winston W. Little, W. Harold Wilson, and W. Edgar More, Applied 
l,ogic (Boston: Houghton Mifflin Company, 1955), p. 1. 
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reached deductively are only as good as the assumptions upon 
which they were made; and that many decisions must be made, 
problems solved, and conclusions drawn in intelligence research 
without the benefit of validated assumptions. Finally, the 
researcher will discover (if he doesn’t know it already) that the 
most difficult type of reasoning he will have to perform is 
induction-a type of reasoning for which hard and fast rules do 
not exist. 2 

Accuracy 

Accuracy has two meanings. In one sense it means pre¬ 
ciseness or exactitude. Thus, the identification of a geographic 
location by use of an eight-digit grid coordinate could be said to 
be more accurate than the same location identified by the 

description “about ten miles northeast of-.” Accuracy also 

means freedom from error, or “conformity to truth.” Both 
interpretations are relevant to intelligence research. For 
example, at one time during the Korean War much confusion 
existed among U.S. military planners because of the failure to 
distinguish between the place-names, Pyongyang, Pyonggang, 
and Pyongan. The assignment of a target the size of a county 
(Pyonggang) provided little guidance for U.S. air strikes, when 
in fact the intended target was Pyongyang, the capital of North 
Korea. 

Potential for error is high in intelligence research, parti¬ 
cularly in those numerous instances in which the researcher 
does not work with “raw” data but instead works with data 
that have been pre-processed by others. For example, an analyst 
working with someone else’s interpretation of a foreign 
language input is subject not only to errors that he may commit 
but also to errors made by the original translator. Even when 
working with raw data, for example, with a foreign language 
newspaper, the researcher may find that there are no direct 


A quick review of principles of logic applied to intelligence can be 
found in Logic for Intelligence Analysts, a series of four programmed 
instructional textbooks prepared for student analysts at the Defense 
Intelligence School. 


50 


I nglish language equivalents for a word or a concept addressed 
.iiul that the best he can do is to approximate the original 
meaning of the work; hence, the potential for error exists. 

Aerial reconnaissance is highly touted as a producer of 
“accurate” raw data. But any intelligence researcher who has 
observed two photointerpreters nearly coming to blows over the 
issue of whether an object was a truck or a tank comes away 
with an uneasy sensation that the potential for error exists even 
when the best data collection techniques are used. Sensors do 
not lie, of course, but humans may lie unintentionally when 
preconceived notions, biases, and strong convictions interfere 
with perception. Unless the researcher is at least aware that he 
may be adding error to what is already unclear, the impact may 
be serious. Again the necessity for developing an awareness of 
subjective factors that influence behavior becomes apparent. 

Necessity for accuracy exists at every stage in the intelli¬ 
gence cycle from collection to dissemination. An ambiguously 
prepared intelligence information report (IR) sets the stage for 
equivocal interpretations, and these in turn yield products of 
dubious quality. The inadvertent deletion of a zero transposes 
10,000 to 4,000, perhaps with disastrous consequences. 

Obviously, the potential for drawing erroneous conclusions 
is the most serious effect of inaccuracy in either data collection 
or data analysis. But the intelligence researcher who is working 
under severe time constraints should also recognize that 
correcting errors, should they be detected, is also a costly and 
lime-consuming operation. 

The fact that much of the input with which the intelligence 
researcher must work is of dubious quality and the fact that 
even presumably “hard” data may have been cunningly altered 
by the other side to mislead are sufficient reasons for the 
researcher to be all the more accurate in the operations he 
performs on the data—operations such as recording, counting, 
citing, interpreting, or quoting. 


51 




























Intellectual Honesty 


In the context of academic research, intellectual honesty 
usually relates to such things as attributing information to 
appropriate sources and giving recognition to others who may 
have contributed significantly to an effort-in short, giving 
credit where credit is due. 

In the context of intelligence research, however, these forms 
of intellectual honesty may be largely irrelevant except, of 
course, in those instances when a research product is submitted 
in fulfillment of academic requirements, for example, at the 
Defense Intelligence School. 

But the other sense of intellectual honesty—accepting 
information that runs contrary to prevailing opinion—is critical 
in intelligence research despite the fact that the analyst may be 
under pressure to support a particular position. 3 Furthermore, 
although it is easy to pay lip service to intellectual honesty, the 
researcher is tested severely in those instances in which his 
entire research effort is jeopardized by the last-minute discovery 
of a bit of information that would tend to refute his main 
thesis. It is an unusual man, indeed, who would not entertain 
even momentarily the idea of conveniently “losing” the 
unpropitious discovery. But in intelligence research, more than 
egos or reputations are at stake; consequently, intellectual 
honesty is imperative even if it means that an entire project has 
to be scrapped and started anew. As Barzun and Graff so aptly 
put it, “Elsewhere honesty may be the best policy, but in 
research it is the only one.” 4 


3 For other ramifications of this problem, see Jack Zlotnick, National 
Intelligence (Washington, D.C.: Industrial College of the Armed Forces, 
1964), page 23. 

4 Jacques Barzun and Henry F. Graff, The Modem Researcher (New 
York: Harcourt, Brace & World, Inc., 1970), page 60. 
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Open-Mindedness 


Open-mindedness relates to one’s receptivity to the argu¬ 
ments of others. It has been stated that whereas several geese 
constitute a gaggle, and several cows constitute a herd, several 
intelligence analysts or researchers constitute an argument. 
Plight people often have strong egos, and researchers deeply 
immersed in their subject matter think (often justifiably so) 
that they have better insights into their problem areas than 
others. But this deep immersion is often the very thing that 
precludes the researcher from seeing the real issues. Anthropolo- 
gists have discovered, for example, that it is very useful to have 
a “naive” observer accompany a field researcher to ensure that 
the trained specialist does not overlook the obvious in his 
concentration on the more subtle manifestations of culture. 

Open-mindedness implies that one is receptive to others’ 
opinions and interpretations-for example, alternative hypo¬ 
theses which could account for a certain condition or event 
even though these interpretations may run contrary to one’s 
own position. It should be noted that receptivity does not mean 
an abject acceptance of everyone’s countervailing opinions. At a 
certain point in every research effort decisiveness is essential 
and a position must be taken if the effort is to be completed. 
Put as a safeguard against committing one’s self prematurely to a 
position that may later become untenable, it behooves the 
researcher to be receptive to alternatives when there is still time 
to take appropriate action. 

Skepticism 

Skepticism is absolutely essential in intelligence research. 
That most countries withhold certain types of information 
makes the information that is released all the more suspect. 
Experience is the best guide for the researcher for attributing 
credibility to a source. Certain countries’ “public announce¬ 
ments” are so systematically misleading that a researcher can 
nearly always assume the opposite of what is reported. Certain 
types of informants believe their worth as sources of 
information is directly proportional to the significance of the 
information they provide. Consequently, their input is 
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exaggerated and distorted. Many would-be sources are 
notoriously poor observers, particularly when they purport to 
observe an object or an activity with which they have no 
familiarity. 

But skepticism may work to the detriment of the re¬ 
searcher, and a distinction should be drawn between a 
‘‘healthy” skepticism and (for a lack of a better term) an 
“unhealthy” skepticism. The former type of skepticism is 
applied situationally: the latter type is applied universally. 
Thus, a hardheaded skeptic would discount categorically any 
statement from a provincial radio broadcasting station in 
Hsi-ning which described problems between the People’s Libera¬ 
tion Army and the Chinese Communist Party simply because it 
is uncharacteristic of communist countries to admit publicly the 
existence of internal problems. 

In academic research, one normally assumes that his 
information is reliable until he finds evidence to the contrary. 
In intelligence research, the opposite is true, and this generaliza¬ 
tion holds as much for input from controlled sources as it does 
for input from uncontrolled sources. 

Detach ment 

The intelligence researcher who becomes involved 
emotionally with the subject of his investigation soon loses 
objectivity. One of the authors noted this lack of detachment in 
a number of young military analysts who were examining North 
Korean, North Vietnamese, and Communist Chinese pro¬ 
paganda that was directed against the United States. Despite 
admonitions to the contrary, many of the young analysts were 
not viewing their daily “take” dispassionately, but were in fact 
reacting emotionally. For example, their first impulse in reading 
a particularly vitriolic attack against “U.S. imperialism” was to 
come up with counterarguments to refute the charges. 

Whatever emotionalism that is generated by the researcher 
should take the form of enthusiasm in attacking a particularly 
vexing problem. Intelligence researchers should attempt to 
perceive their activity as a game—a serious game in which two 
opposing sides strive for complementary goals. One side tries to 
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conceal, hide, deceive, or mislead, whereas the other side 
attempts to penetrate the screen of ruses and stratagems in 
order to perceive reality. Emotionalism requires energy that 
should be directed elsewhere for more positive ends. In the case 
of the propaganda analysis just described, what was significant 
were not the contents of the anti-U.S. diatribes (which tended 
to be highly repetitive), but rather that the diatribes were being 
uttered in the First place and with a high degree of regularity, 
these facts tended to be overlooked in the analysts’ emotional 
reaction only to the content. 

It would be unrealistic to expect anyone engaged in an 
activity as serious as intelligence to love his enemy; but a rabid 
hatred of the other side interferes with one’s perception and 
invariably destroys objectivity. 

Patience, Diligence, Perseverance 


Intelligence analysts and researchers who anticipate quick 
solutions to difficult problems are in for a rude awakening. 
Barely do reports which contain succinct summaries or brilliant 
deductions give any indication of the amount of sheer drudgery 
their production entailed. Yet drudgery is more the common 
rule than exception in intelligence research. Dramatic solutions 
to intelligence problems are fairly rare events and the researcher 
who requires this type of positive reinforcement may soon 
become discouraged when he finds that solutions elude him, 
when he discovers that he had been proceeding up blind alleys, 
and when he is forced to reexamine for the hundredth time the 
same material with which he had been working for months. 

Tycho Brahe, Kepler, and Copernicus made thousands of 
observations for many years before they were in a position to 
publish their discoveries. Darwin pondered, reflected and 
reexamined his findings for years and, had not another biologist 
been ready to announce findings similar to his, Darwin probably 
would have procrastinated even longer in publishing his theories 
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on evolution. 5 Admittedly, these examples provide little solace 
to the intelligence researcher working under severe time 
constraints. But the urgency of the problems requiring solutions 
makes patience, diligence, and perseverance all the more 
important. 

Imagination 

The trait that distinguishes best between the very adequate 
researcher and the outstanding researcher is imagination. And 
unlike other traits which can be acquired by training, this trait 
is indeed difficult for many researchers to develop. Imagination 
in the context of research does not refer to flights of fanciful 
musings, daydreams, and the like (although serendipitous 
spinoffs sometimes occur in this manner). Imagination in the 
context of research relates to releasing a kind of creative energy 
that provides unorthodox (but effective) approaches to prob¬ 
lems which seemingly defy solution. Sometimes this creativity 
manifests itself in an ingenious approach for data collection. In 
other instances this creativity takes the form of assembling data 
in such a manner that new implications become apparent. 

Margaret Mead, in discussing the problem of studying 
cultures at a distance (cultures to which the researcher is denied 
access, or cultures which no longer exist^), describes the 
importance of imagination in anthropological research: 


5 Data collection for Darwin’s epochal work, On the Origin of Species 
by Means of Natural Selection , began twenty-eight years before the book 
was published. (Origin of Species also included the results of later 
observations, of course.) This is a classic example of an instance in which 
the data collection phase occurred before the problem definition phase of 
a research project. It is interesting to speculate whether Darwin could have 
published his results many years earlier had his data collection efforts on 
the voyage of the Beagle been guided by formal hypotheses. 

^Conditions which are not uncommon to the researcher in Sociologi¬ 
cal Intelligence. 
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For the study of culture at a distance, however, 
additional capacities are needed, because in every case 
the research worker who uses a particular kind of 
material-interviews, films, art forms, games, slang, and 
so on—is required to go beyond his or her source 
material, to delineate in terms of a larger whole, the 
culture, that is, the total shared, learned behavior of the 
members of the group or society or period being studied. 

In historical studies this has been called the historical 
imagination-the ability to reconstruct from a set of 
parchments, epitaphs on gravestones, lists of purchases 
by a steward for a manor, or of expenditures for the 
costumes worn in a morality play, the life of a long-past 
period as a whole. For such reconstruction, the student 
must be able to move from one set of clues to another, 
so that if he has a painting that shows the costumes 
worn in a period, a list of expenditures for stuffs, a list 
of the foods that were sold in the shops, a few bars of 
religious music, a knowledge of the climate, a calendar, 
he will be able to see, hear , and smell a thronged 
medieval street down which a Whitsunday procession 
passed. Only so, by fitting together separate sets of clues 
or traces into a reconstructed living whole, can the parts 
be made meaningful. 7 

Obviously, the admonition to go beyond one’s source 
material must be heeded with discretion, and certainly no 
researcher is justified in making assertions that are not 
supported by facts. But facts alone do not constitute intelli¬ 
gence research. The critical question in all research is What do 
the facts mean? And this is where imagination plays its 
important role. For example, intelligence problems which 
cannot be solved directly are often “solved” by analogy. Yet, 
selecting or constructing an appropriate analogy requires 


7 Margaret Mead and Rhoda Metraux, ed., The Study of Culture at a 
Distance (Chicago: The University of Chicago Press, 1953) p. 11. 
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imagination. One of the techniques employed by “futuro¬ 
logists” and by war gamers is to construct “alternative futures.” 
Constructing “alternative futures” is a creative act involving 
speculations in the form of “What would happen if. . . ?” 

An outstanding example of the role that imagination can 
play in the practical problems of data collection in the social 
sciences are the “unobtrusive measures” described by Eugene J. 
Webb et al. For example, unable to measure directly the interest 
shown by spectators to various displays at a convention, one 
ingenious social scientist counted the number of nose prints on 
the various glass display cases. In similar applications, the 
researchers measured the amount of wear on floor tiles in front 
of various displays to determine which displays attracted the 
greatest number of viewers. 8 Admittedly, some of the techni¬ 
ques produce only “squishy” data (as opposed to “hard” data), 
but in many cases even “squishy” data are sufficient to permit 
decisions to be made. 9 

Imagination sometimes takes the form of a “hunch,” that 
humble precursor of the more eloquent hypothesis. Washington 
Platt, no stranger to intelligence research, defines a “hunch” as 
follows: 


A scientific hunch is a unifying or clarifying idea which 
springs into consciousness suddenly as a solution to a 
problem in which we are intensely interested. In typical 
cases, it follows a long study but comes into conscious- 
_ ness at a time when w e are not consciously working on 

^Eugene J. Webb et al, Unobtrusive Measures: Nonreactive Research 
in The Social Sciences (Chicago: Rand McNally & Company, 1966). 


9 Lest the reader take affront at the use of slang, “squishy” has been 
defined as “ . . . without any well defined mathematical formulation that 
unambiguously captures the substantive problem.” Ralph E. Strauch, in 
his report, A Critical Assessment of Quantitative Methodology as a Policy 
Analysis Tool (Santa Monica, California: The Rand Corporation, 1974), 
has given a new respectability to a term that has been used by social 
scientists for years. 
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the problem. A hunch springs from a wide knowledge of 
facts but is essentially a leap of the imagination , in that 
it goes beyond a mere necessary conclusion which any 
reasonable man must draw from the data at hand. It is a 
process of creative thought.^ 

Hunches, insights, intuitions, alternative solutions, alterna¬ 
tive approaches, the ability to envision an ideal (e.g., a source of 
data, or an end product) are all manifestations of imagination 
which have very practical applications in intelligence research. 

The intelligence researcher “cannot lay in stock” of 
imagination, but, as Barzun and Graff point out, he can learn 
techniques for releasing whatever imaginative powers he has. 
Constraints to imagination and creativity appear to be largely 
self-induced. Because of fear of error, fear of ridicule for 
suggesting unorthodox approaches or conclusions, and because 
of excessive concentration on a problem, many researchers 
consciously or unconsciously limit their own creativity. 

Platt discovered that many successful researchers had their 
most creative insights just before falling asleep, or when they 
were engaged in some activity unrelated to their investigation, 
when they were listening to music, or when they were just plain 
relaxing; and there may be some truth to the story of 
Archimedes’ formulating his theories of buoyancy while in the 
bath. But before the naive researcher sets out to create those 
conditions most conducive to creativity, it is important to note 
that, as Platt points out, “Most [researchers] reported that 
hunches came during periods of apparent idleness—following, 
however, long periods of intensive work.”* * 


^Washington Platt and Ross A. Baker, “The Relation of the 
Scientific ‘Hunch’ to Research Journal of Chemical Education, October, 
1931, p. 1975. (Italics not in the original.) 
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Summary 


V' ( ertain human traits are essential for anyone attempting to 
perform acceptable research: reasoning ability, accuracy, 
intellectual honesty, and open-mindedness. These are necessary 
1 but not sufficient) traits for any researcher. 

For the intelligence researcher, additional traits are re¬ 
quired. These additional traits include: skepticism, a sense of 
detachment, patience, diligence, perseverance, and imagination. 

F 1 Imagination is especially necessary in intelligence research in 
order to develop innovative techniques for collecting and 
analyzing data that the other side wishes to deny to outsiders. 
Imagination is also necessary for generating various alternative 
hypotheses or theories to account for conditions or events- 
hypotheses which could be tested at a later date. 


CHAPTER VII. 

TYPES OF INQUIRY AND THE 
NATURE OF PROOF 


What is now proved was once only imagined. 

William Blake 

Under a cover of low, thick clouds, a carrier strike force in 
(he North Pacific launched an air strike consisting of bombers, 
torpedo planes, and fighters. The destination was Pearl Harbor; 
lhe objective, units of the U.S. Pacific Fleet. The approach to 
Oahu was completely undetected, and the result of the air strike 
was the theoretical destruction of the whole Pacific Fleet. The 
destruction was theoretical because the air strike was part of a 
U.S. naval war game conducted (ironically) Sunday, February 7, 
1932. The activity was a simulation. It was a representation, or 
a model , of a genuine attack that could occur in the future. 

Some years after the exercise, General Billy Mitchell, one of 
the observers of the game and a man noted for his outspoken¬ 
ness, said: 

If the President [Roosevelt] can be made to see that the 
trouble will start with Japan, perhaps we’ll have more 
planes in the Philippines and Hawaii. For years he’s had 
the idea that a war in the Far East would be impractica¬ 
ble and that an attack upon us by Japan is inconceivable. 

That’s Navy thinking. The Japanese will not politely 
declare war. Hawaii, for instance, is vulnerable from the 
sky. It is wide open to Japan . . . and Hawaii is swarming 
with Japanese spies. As I have said before, that’s where 
the blow will be struck-on a fine, quiet Sunday 
morning. 1 

* Quoted by Emile Gauvreau in The Wild Blue Yonder (New York: E. 
P. Dutton & Co., 1944), pp. 169-171. 
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The prediction (or better, prophecy) made by General 
Mitchell was founded on inspiration, judgment, and imagina¬ 
tion. His prophecy was not a prediction in the scientific sense of 
the term because it was not based on observations of recurring 
phenomena. Many “predictions” in intelligence, of necessity, 
cannot be made on the basis of recurring phenomena simply 
because the phenomena of greatest concern are those which 
have not yet occurred, or, if they would occur, would not be 
repeated. The most obvious example, of course, would be a 
nuclear attack by one of the major powers against the other. 

This chapter is about descriptive and predictive research 
because these two kinds of activities constitute the major 
portion of intelligence research and analysis activities. 

This chapter will discuss how the nature of proof differs 
between descriptive and predictive research. It will be shown, 
for example, that much of the research which purports to be 
predictive is really a variation of descriptive research. This 
chapter will also discuss models—of which the Naval war game 
conducted in 1932 was an example—and will show the 
relevancy of model validation to intelligence research, and 
particularly to predictive research. 

At the outset it should be noted that the distinction 
between predictive research and descriptive research is a 
nebulous one. For example, most valid predictions are based on 
descriptions of past activities or observations of recurring 
phenomena. 

As the term will be used throughout this work, predictive 
research will refer primarily to the activities involved in 
developing and testing models that foretell what the outcomes 
will be, given the interaction of certain variables. 

Descriptive Research 

As its name implies, descriptive research relates to activities 
which attempt to reconstruct, explain, interpret, account for, or 
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describe a phenomenon 2 * . This type of inquiry may be used 
when unique phenomena or recurring phenomena are examined. 
An example of a unique phenomenon would be a one-of-a-kind 
event, an object, or an individual. Recurring phenomena, of 
course, are activities or events which occur many times, for 
example: missile firings, communications, and tactical maneu¬ 
vers. 

Like all research, descriptive research attempts to provide 
new information—information which might support earlier 
conclusions or cause entirely new conclusions to be formulated. 

In intelligence research, new information may require that 
new estimates be prepared. In scientific research, new informa¬ 
tion may require that new principles be formulated when 
existing principles no longer adequately account for facts. 

Predictions, if any, evolving from only descriptive research 
tend to be “squishy” in the sense that they do not lend 
themselves to “. . .any well defined mathematical formulation 
that unambiguously captures the substantive problem.” 4 An 
example of a “squishy” prediction evolving from descriptive 

research is the statement: “By the year- ■, the Soviet 

Union’s missile arsenal will consist of ICBM s and 

MRBM’s.” Clearly, this type of prediction is a far cry from the 
prediction of solar eclipses, from the prediction of movements 
of subatomic particles, and from the prediction of meteorolog¬ 
ical phenomena. 


2 Some writers would classify explanation as a separate type of 
research. However, many “explanations” are merely descriptions of the 
interaction of variables. Explanations of events, for instance, are often 
descriptions of antecedent conditions which purportedly give rise to the 
subsequent events. 

3por example, it became necessary to develop quantum mechanics to 
account for the movement of subatomic particles because these phenom¬ 
ena were not adequately addressed by Newtonian mechanics. 

4 Strauch, A Critical Assessment of Quantitative Methodology. 
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Nature of Proof in 
Descriptive Research 

Descriptive research is distinguished from predictive re¬ 
search on the basis of the techniques used to test an hypothesis 
Very briefly, an hypothesis is a “testable assertion ” 5 or a 
tentative statement that proposes “ . . . causal relations’between 
various sets of facts.” 6 In descriptive research, testing an 
hypothesis usually involves searching for evidence that supports 
or refutes it. An historian (or an intelligence analyst), for 
example, might postulate an hypothesis to account for the 
occurrence of an event. He would “test” his hypothesis by 
searching for information that supported his hypothesis as well 
as for information that would refute it. His hypothesis could be 
said to be “valid” if the evidence obtained (and presumably, all 
of the evidence that existed) supported his hypothesis more 
plausibly than any alternative hypothesis. “Plausibility” in this 
instance would imply possibility, consistency with other facts 
and theories, and it might also imply simplicity. For example in 
cases in which all other things were equal, the simple 
explanation (hypothesis) would be preferred over the complex. 

In those instances in which an historian (or a sociologist or 
an anthropologist) might attempt to formulate a principle based 
on recurring conditions or events,7 he would attempt to test his 
hypothesis by searching for other instances comparable to the 
event under consideration in which certain precedent conditions 
were followed by certain subsequent events. Should he discover 


. 5ja " 1e /. r B - Conant, On Understanding Science: An Historical 

Approach (New York: The New American Library, 1951), p. 38. 

wo JritV' y° Ung ’ Scient ‘f ,c ^cial Surveys and Research (Engle¬ 
wood Cliffs, New Jersey: Prentice-Hall, Inc., 1966), p. 19. 

7 For example, Spenglerand Toynbee. Studies of “national character” 
would be examples of intelligence research that attempt to formulate 
principles of group behavior. 
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comparable instances in which the relationship of precedent to 
subsequent conditions existed, these instances would tend to 
corroborate his hypothesis. The more instances he found that 
corroborated his thesis, the stronger his confidence would be 
that his thesis was valid. The researcher would also search for 
instances in which the same precedent conditions were not 
followed by a certain subsequent event. Should he find such 
instances, his hypothesis would be invalidated. Obviously, the 
historian cannot cause an historical event to occur again; thus, 
he cannot replicate his test. He can only search for, and hope to 
find, conditions similar to the one under consideration. The 
extent to which conditions are comparable is the extent to 
which his confidence in an hypothesis is maintained. 

Predictive Research 

Predictive research also attempts to explain or to account 
for a phenomenon, but in addition, predictive research also 
attempts to formulate new principles (or natural laws) which 
foretell what the effects will be of the interaction of specified 
variables. Principles are fundamental truths or general laws 
based on observed regularities of the external universe. Princi¬ 
ples are the foundation of modern scientific thought because 
they permit predictions to be made with a high degree of 
accuracy. Predictive research, of course, is most applicable to 
recurring phenomena. Disciplines which best exemplify predic¬ 
tive research in its purest form are the physical sciences. 

Nature of Proof in 

Predictive Research 

In contrast to the historian and the typical intelligence 
researcher who must search for whatever evidence exists to test 
his hypothesis, the physical (or experimental) scientist can 
manipulate independent variables in his laboratory according to 
plan. As such, he can create the very conditions upon which his 
prediction (hypothesis) is based, and he can compare almost 
immediately the actual outcome of his manipulation of 
conditions (variables) with his predicted outcome (his 
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hypothesis). * * 8 If the actual outcome of the experiment com¬ 
pares favorably to the predicted outcome, the scientist’s 
hypothesis is validated. 9 

The test procedure used for validation in its broadest sense is 
the experiment. Again, in its broadest sense, an experiment may 
range from a mental exercise of eliminating alternative hypoth¬ 
eses (as an historian or an intelligence researcher might do) to a 
laboratory manipulation of controlled variables. 

Rarely can the intelligence researcher perform controlled 
experiments, but often he can make numerous observations of 
the same kinds of phenomena. Although not as good as 
observations of controlled situations, these observations never¬ 
theless can be used as inputs to predictive models. 

For the sake of comparison, the scientist’s procedures were 
simplified. The point that should be noted, however, is that the 
scientist can replicate his experiment dozens of times. Perhaps 
more important, other scientists can also replicate the same 
tests. Herein lies the difference between “squishy” predictions 
evolving from descriptive research of unique events, and “hard” 
predictions evolving from many repetitions of the same tests by 
independent researchers. The former type of prediction is 
usually based on a very limited sample—sometimes a sample of 
only one. Rules for accepting hypotheses in these cases are 
different from rules for accepting hypotheses based on a sample 


^Immediate as opposed to the length of time required to test 

Malthus’s theory of the world’s population outstripping its food supply, 

and Spengler’s theory of the decline of the West. 

9 This is an oversimplification, of course. In reality, the scientist tests 
the “null hypothesis”-the hypothesis which says in effect that any results 
obtained could have been obtained by chance. When data do not support 
the null hypothesis, it is rejected, and the rejection of the null hypothesis 
is tantamount to the acceptance of the original hypothesis. 
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of say, one hundred. In the former case, possibility and 
plausibility enter into the picture, as well as the persuasive 
power of the person formulating the hypothesis-his knowledge, 
experience, and past history of successful predictions. In t e 
latter case, acceptance or rejection of an hypothesis is dictated 
almost entirely by statistical tables. 

Although the nature of proof differs between descriptive 
and predictive research, this does not imply that one type ot 
proof necessarily approches “truth” more closely than the 
other. Both types of proofs are appropriate in their respective 
domains and equally inappropriate in the other. Gregor Men¬ 
del’s principles of genetics would have been meaningless 
without supporting statistics, but it is difficult to see how 
Edward Gibbon’s explanation of the decline and fall of the 
Roman Empire could have been enhanced by any experimental 
treatment. Courts of laws also have their own rules oi 
evidence” for establishing “proof.” For the sake of accuracy, it 
is better to reserve the term “proof’ for mathematics and to use 
the terms “validity” and “validate” in the context of hypothesis 
testing. “Proof’ denotes certainty which is contradictory to the 
tentative nature of hypotheses and theories. 
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Theory Validation in a 
Conflict Situation ’ 


., 6 ° re ,- , the reader concludes that experimentation and 

theory validation have nothing to do with intelligence and 
intelligence research, it would be well to consider what takes 
place when one nation attempts to ferret the operational 
pattern of another country’s aircraft early warning system. The 
typica method involves sending aircraft loaded with electronic 
gear close to or across another country’s border and to record 

e™ mber ’ types ’ locat ions and characteristics of the radars 
that the enemy activates. 

Having executed this maneuver a number of times, a picture 
of the other side’s aircraft early warning network evolves 
Invariably gaps exist in the picture. These gaps are filled initially 
with inferences or calculated guesses of the types of devices that 
s ou be covenn 8 a certain area based on observed regularities 
in o her areas in the past. These inferences are testable 
hypotheses, and if time permitted, attempts would be made to 
provoke the enemy into turning on the radars in the “untested” 
territory so that the hypotheses could be validated. If it weren’t 
for the fact that the enemy had the option of not turning on his 
adars (thus, was not bound by any natural law), this type of 
operation would qualify as predictive research in the most 
rigorous sense of the word. 

Observed regularities of any enemy activity create the 
framework in which numerous kinds of quasi-predictive re¬ 
search activities can be performed. Recurring activities as 
mentioned previously, include standing operating procedures in 
communication, similar patterns in conducting field operations 
and similar patterns of weapons systems research, test, and 
development. But all of these activities are governed by choice 
on the enemy s part, and not by some inviolable law of nature. 
Hence, the strength of a prediction made on the basis of 
recurring activities in the past is never so strong in human 
affairs as it is in, say, the physical sciences. Confounding the 
issue is that sometimes the enemy deliberately maintains a 
pattern of activities to conceal some other activity or to 
mislead or deceive. 
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Despite inherent limitations, validating predictions and 
attempting to establish “working principles” are fairly common 
activities in intelligence and intelligence research, and considera¬ 
tion of the nature of proof is by no means a mere academic 
exercise. 

Models 

A concept that evolved in the physical sciences, but one 
which has application in many other non-scientific investiga¬ 
tions and particularly in intelligence analysis and research, is the 
concept of the model. In its simplest sense, a model is an 
abstraction or representation of reality. For example, chessmen 
are models (representations) of combatants, and a chess game is 
a model of conflict; a chemical formula is a symbolic 
representation (model) of a substance; a model airplane is a 
miniature representation of the real thing (with many parts 
missing, of course); the naval war game described at the 
beginning of the chapter was a model of the actual conflict; and 
game theory is a model of political, military, or economic 
conflict, again with irrelevant parts deleted. Scientific theories 
are also models. The advantage of models is that they enable 
one to predict outcomes by manipulating only symbols rather 
than elements of the real world. The most common type of 
symbol manipulation for predictive purposes is simulation. 10 

Models and the 
Scientific Method 

Whenever a researcher formulates an hypothesis that at¬ 
tempts to account for the interaction of variables, he is 
constructing a model. The formulation of the model is a 
creative act; hence, the necessity for imagination discussed 
earlier. The quality of a model is a function of how well it 
accounts for the interaction of variables. In scientific models, 
quality is the extent to which the model predicts accurately. It 
is the act of determining how well a model explains or predicts 
that constitutes the scientific method. 


^Models, games, and simulations will be discussed further in 
Chapters XVII and XVIII. 
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Very briefly, the scientific method involves the following 
steps: 1) the researcher makes observations either directly or 
vicariously by studying reports of earlier observations and 
measurements; 2) the researcher formulates (postulates) a 
model which attempts to account for the phenomenon ob¬ 
served; 3) the researcher subjects his model to a test (the 
experiment). He checks his measurements, examines his logic 
(which is reflected in the model he postulated as well as in the 
design of his experiment), and compares results obtained against 
results predicted by his model; 4) if the results of his test are 
comparable to the results predicted by the model, his model 
may be said to be valid. If the obtained results are inconsistent 
with his model, and if the researcher ascertained that he made 
no error in measurement or observation, he may either revise his 
model and subject it again to the same series of tests, or he may 
simply discard the model and attempt to formulate a new 
hypothesis. The steps of the scientific method are shown in 
Figure VII-1. 

It should be noted that the “basic” scientist is primarily 
concerned with establishing the quality of the model: he is not 
concerned with making specific predictions. Once models are 
validated, however, they can be used for making any number of 
predictions at any future time. 11 The “applied” scientist (like 
most intelligence researchers), on the other hand, is concerned 
with making specific predictions by applying the model in real 
situations. 

The Scientific Method 
in Nonscientific Research 

An historian or an anthropologist who postulates an 
hypothesis to account for an historical event or to describe the 
uses of a cultural artifact also constructs a model. Unlike the 
scientist’s model, validation of an historian’s hypothesis does 
not involve comparing actual events or conditions with pre¬ 
dicted events or conditions. The canons or criteria for accepting 


* 1 Provided, of course, that the observed relationships among variables 
do not change. 
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HE POSTULATES » MODEL (SUTED IN THE FORM OF 
AN HYPOTHESIS...) 


HE TESTS THE MODEL (CONDUCTS AN EXPERIMENT) 


HE COMPARES HIS OBTAINED RESULTS WITH HIS 
PREDICTED RESULTS. 


Figure VIM. Model Validation and the Scientific Method 12 


12 After Marshall Walker, The Nature of Scientific Thought (Engle¬ 
wood Cliffs, N.J.: Prentice-Hall, Inc., 1963), p. 5. 
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an hypothesis, as pointed out earlier, are instead a matter of 
how well existing data “fit” the theory. But the point that 
should be remembered is that the steps in the scientific method 
are as relevant to nonscientific research as they are to validating 
a scientific predictive model. And with certain qualifications, 
the scientific method is a most appropriate methodology for 
intelligence research as well. 

Typically, 'intelligence research or analysis begins when the 
researcher makes observations. Unlike the physical scientist who 
may observe his phenomena directly, however, the intelligence 
researcher or analyst usually makes his observations vicariously; 
that is, he studies documents, reads intelligence reports, studies 
photographs, or interviews participants or other observers. On 
the basis of his observations, he postulates tentative explana¬ 
tions of what is taking place, or why something is occurring. 
These initial tentative explanations are called working hypothe¬ 
ses. These working hypotheses are examined in terms of 
possibility and plausibility, and the hypotheses which seem most 
plausible are retained for further validation. 

The retained hypotheses are tested against new input, and 
the hypothesis which is supported best by the new input (i.e., 
most consistent with the new input) becomes the basis for the 
researcher’s conclusions. 

In general, this procedure is not too dissimilar from the 
methods used by the physical scientist. But there are a number 
of important differences that may occur in intelligence research. 
For instance, whereas the experimental scientist limits his 
hypotheses to one, the intelligence researcher, in order to 
exhaust all logical possibilities more quickly, will attempt to 
postulate as many possible and plausible hypotheses as he can, 
and he may very well test several hypotheses simultaneously. 13 
He is forced to do this because the variables cannot be 


l^In reality, hypotheses are never tested singly. At a minimum, there 
is always a null hypothesis for any hypothesis postulated, as pointed out 
earlier. 
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replicated in the real world as they can be in the laboratoiy. 
Whereas the physical scientist can methodically test one 
hypothesis, the/ replicate the variables and test another 
hypothesis, and so on, until he has exhausted a 1 of his 
hypotheses or until he has validated one, ^the intelligence 
researcher cannot reconstruct the situations Actors- change 
conditions change, and above all, time changes. Although it 
might be possible to recreate the physical conditions surround¬ 
ing an event, it is impossible to recapture the event without 
taking into account the learning or experience that was gained 
by the participants since the original event occurred. 

Ideally, hypotheses should not be tested against information 
used to formulate the hypotheses initially, smg e> sani® 
information for both hypothesis generation and hypotheSB 
testing may give rise to circular reasoning. However, in historical 
research it may not be possible to obtain additional informa- 
lion" as’such the test of an hypothesis is simply the extent to 
which the existing facts fit the theory. 

Another difference between intelligence research and re¬ 
search that relates to establishing scientific principles is the ro e 
that prediction plays in validation. A scientific hypothesis is 
said to be valid only when a predicted event occurs. However 
much intelligence research and analysis is P erformed f « r th . e 
very purpose of precluding an event s occurring, hence, it is 
often impossible to use an actual event for the purposes o 
validating a prediction. 

Finally, research conducted to establish scientific principles 
seeks to uncover relationships that permit the prediction of 
outcomes of classes of events. 15 Intelligence research, on the 
other hand, when it addresses prediction, is more concerne 
with predicting specific events. 

14 In hypothesis formulation one attempts to “fit” a theory to the 
facts. In hypothesis validation, however, the test is how well the facts i 
the theory. 

15As an oversimplified example, theories of gravity apply as much to 
apples as they do to ballistic missiles. 
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The general plan that would be followed for validating 
hypotheses in intelligence research is shown in Figure VII-2. 

Predictive Model Validation 
in Intelligence Research 

Predictive model development and validation make up a 
significant portion of intelligence research activities. Economic 
Intelligence researchers and analysts, for example, employ 
predictive models quite frequently, and model validation may 
constitute a major portion of their analytical or research 
activities. For instance, agricultural economists may devise 
algorithms (or formulas) for predicting grain yields given certain 
variables such as acreage under cultivation, availability of 
fertilizer, growing season, and precipitation. The initial 
algorithm is a model—a model which must be validated and 
refined by comparing predicted yields with the actual yields. In 
terms of armament production, economists may again devise an 
algorithm for estimating the production capabilities of a 
manufacturing facility, given variables such as the number of 
furnaces, storehouses, floor space, and available transportation 
nets. 

In Transportatio n Intelligenc e, models or algorithms are used 
for estimating the amount of materiel that can be moved over 
selected rail lines, given such variables as transloading facilities, 
availability of single-or double-tracked lines, availability of 
rolling stock and locomotives, train length, and so on. Manipu¬ 
lating these variables by means of computer simulation permits 
analysts to estimate (predict) how much could be carried under 
varying conditions and configurations. 

Sometimes the intelligence community enjoys a windfall 
and an already validated model of the enemy (or his operations) 
becomes available for operational application. An example of 
this took place in World War II when U.S. intelligence analysts 
were able to estimate the size of the Japanese garrison on Betio 
(Gilbert Islands) based on the number of privies constructed out 
into the lagoon. The “model” that the analysts used were 
vr captured Japanese documents containing army doctrine that 
specified the required ratio of sanitation facilities per number of 
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intelligence researcher makes observations 

(DIRECTLY OR VICARIOUSLY). 


HE POSTULATES AS MANY ALTERNATIVE 
HYPOTHESES (MODELS) AS HE CAN. 


HE SUBJECTS THESE INITIAL HYPOTHESES TO 
TESTS OF POSSIBILITY.LOGIC,PLAUSIBILITY, 
CONSISTENCY.SIMPLICITY, AND SO ON 


HE SELECTS THOSE HYPOTHESES WHICH WITH¬ 
STOOD THE INITIAL SCREENING AND SUBJECTS 
THESE HYPOTHESES TO FURTHER TESTS... 


IF THE NEW DATA DO NOT FIT THE HYPOTHESES, 
THE RESEARCHER POSTULATES NEW HYPOTHESES 
AND REPEATS THE CYCLE. 


IF THE DATA FIT THE HYPOTHESES, THE 
RESEARCHER DECIDES WHICH HYPOTHESIS IS 
SUPPORTED BEST. HE MAY RANK ORDER THE 
HYPOTHESES OR SIMPLY DISCARD THE HYPOTHESES 
WHICH ARE NOT SUPPORTED BY THE AVAILABLE 
DATA 

HYPOTHESES BEST SUPPORTED BY AVAILABLE DATA 
BECOME BASES FOR CONCLUSIONS OF THE STUDY.. 


Figure VII-2. Hypothesis Validation in Intelligence Research 
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men , Th . e estlm ated size of the garrison missed the actual 
number by only 64 men (out of 4715). 16 

Semantic Problems of 
Technical Terms 

Language plays tricks on the unwary. For example, because 
an investigation is nonscientific does not (or should not) imply 
that an investigation is unscientific. Science relates to activities 
which attempt to establish verifiable general laws. But in the 

from anv ° rganized knowledge, it is not distinct 

from any other discipline such as history, anthropology or 

^ “sdeicS” S ? h a l ter °f intereSt ’ ma °y disciplines classified 
of m ran f h6S that deal with the establishment 

of natural laws or principles in only the most remote manner 

Taxonomy in botany, zoology, and geology are cases in point.' 

ese disciplines are essentially descriptive, not predictive. And 
whether or not the most rigorous experimental scientist admits 
it, all science started as descriptive research. 

_i But °" e of . the more unfortunate tricks that language may 

the “scie^tifi!" 1111 T d ,,f Searcher is that ^ might suggest that 
• +■ method is a technique used only by the 

scientist. This, of course, is not true. The scientific method can 

theT ^ a r range ° f activities fr »m predicting movement of 

Th aCe \t?. P erformin S maintenance on a motor¬ 
cycle. The scientific method is not a body of arcane 
procedures that unerringly reveal truths. The scientific method 


iv !^r U rf E1 , i0t M ° riS0n ’ History of United States Maval Operations in 
V I °‘, VII; Aleutians > Alberts and Marshalls, June 1942-April 
1944 (Boston: Little, Brown and Company, 1961 ), p. 149 . 

1 J For a nontechnical discussion of the scientific method applied to 
everyday problems, the reader is advised to peruse Robert M. Pirsie’s Zen 

Y v wm" °{Z° t0rCycle Maintenance: An Inquiry Into Values (New 
York. William Morrow & Company, Inc., 1974)-a book that, in reality 
says virtually nothing about Zen, and very little about motorcycle 
maintenance. Despite the title, the book is essentially a critique of rational 
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is simply a tried and tested set of procedures for seeing how 
well a theory works. As such, the scientific method can be 
applied to any type of inquiry in which theories are developed 
and tested. In this sense, the scientific method is a most 
appropriate set of procedures for intelligence research. 

Summary 

D> Predictive research and descriptive research are the two 
types of activities which constitute the major portion of 
intelligence research and analysis activities. 

t> Descriptive research attempts to reconstruct, explain, inter¬ 
pret, account for, or describe a phenomenon. “Proof’ in 
descriptive research involves searching for data that support or 
refute an hypothesis or theory. ^ 

> Predictive research attempts to formulate new principles (or 
natural laws) which would enable the researcher to predict, 
anticipate, or foretell what the results of the interaction of 
variables will be. “Proof’ in predictive research involves 
comparing a predicted outcome against a real outcome. 

t> As the term is used in this book, predictive research will 
relate primarily to the process of developing and validating 
predictive models. 

> Procedures for developing and validating predictive models 
are sometimes referred to as the “scientific method.” 

fc> The scientific method involves four steps: 

— The researcher makes and records observations 
— He formulates a conceptual model which attempts to 
account for the phenomenon (postulates hypotheses) 

— He tests the model (in laboratory situations he would 
experiment) 

— He compares the predicted outcome from his model 
with the real outcome. If the results are comparable, the 
model is said to be valid 


77 












> The scientific method, with modifications, is applicable to 
nonscientific investigations as well. 

t> Theory validation in intelligence research differs from 
theory validation in scientific research in the sense that many 
hypotheses may be tested at the same time. Furthermore, rarely 
can the intelligence researcher control variables in the sense that 
a scientist in a laboratory can. For the greater part, intelligence 
researchers must search for conditions or events that would 
permit them to test a theory. Physical scientists, and other 
experimental scientists can create the conditions needed to test 
a theory. 
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CHAPTER VIII 

THE RELATION OF INDUCTION AND DEDUCTION TO 
THEORY BUILDING IN INTELLIGENCE 
RESEARCH 


The basic postulate of all science states that nature is, to 
some degree, predictable; this postulate is merely an 
inductive inference based on man’s experience. He has 
found that attempts to predict have been successful 
enough to be useful in the struggle for survival The 
postulate merely serves notice that man expects to 
continue the efforts to make predictions as long as they 
are useful to him 

Marshall Walker 

On November 3, 1943, Adolf Hitler issued an order for the 
defense of the western approaches to Festung Europa. All 
indications, according to Hitler, pointed to an Allied invasion of 
France by 1944. The most likely place for the invasion was the 
Strait of Dover. Defenses were to be strengthened to the 
maximum in the areas along the strait and Feldmarschalls Gerd 
von Rundstedt and Ervin Rommel were assigned the mission of 
defeating the enemy invasion forces. 

Von Rundstedt, like Hitler, believed that the Allied invasion 
would take place between Le Havre and Dunkirk, despite the 
concentration of German coastal defenses, because an invasion 
here would permit the Allies to deploy in the plains of Picardy 
and drive eastward to the Ruhr. Very few German generals 
thought that the Allies would be foolish enough to risk getting 
bogged down in Normandy. Furthermore, the launching plat¬ 
forms for the V-l and V-2 bombs and rockets that Hitler was 
preparing to use in mass attacks on England were close to the 
Strait of Dover. 1 In short, it was only “logical” that the Allies 

1 Samuel Eliot Morison, History of United States Naval Operations in 
World War II, vol. XI: The Invasion of France and Germany, 1944-1945. 
(Boston: Little, Brown and Company, 1962), pp. 39-49. 
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would launch their inevitable invasion across the Strait of 
Dover, the classic invasion route for Alfred the Great, and the 
route specified in the aborted plans of Napoleon Bonaparte and 
later Hitler, himself, in 1940. 

The Allied invasion did not take place across the Strait of 
Dover, of course. On the basis of assumptions, generalizations 
based in part on elaborate deceptive measures of the Allies, and 
faulty deductions, the German High Command made one of the 
most fateful decisions of the war. The errors made by the 
German High Command were errors of induction. Unfor¬ 
tunately, for commanders in the field, for intelligence analysts 
at their desks, and for intelligence researchers in general, there 
are no safeguards against drawing faulty generalizations—no 
rules to follow, and no exercises to “strengthen” one’s mental 
faculties so that correct generalizations necessarily follow from 
observations. 

This chapter will discuss the reasoning processes involved in 
intelligence research and analysis. More specifically, the chapter 
will discuss the reasoning processes as they relate to different 
steps in the scientific method. In this chapter, the term 
hypothetico-deductive method will be used in place of scientific 
method. Both terms mean the same thing. Although hypo¬ 
thetico-deductive method is not as familiar as the term scientific 
method, the term describes the problem-solving process more 
accurately, and it avoids the connotation that a procedure may 
be limited only to scientific inquiry. 

Initially the chapter will discuss the mental processes by 
which vague concepts are translated into hypotheses which can 
be tested. Then the chapter will discuss the mental processes 
involved in testing hypotheses. With respect to formulating 
hypotheses, however, a word of warning is appropriate at the 
outset: there is little advice that can be given to the researcher 
at the hypothesis formulation stage of his inquiry. Much of the 
activity that precedes the formulation of testable hypotheses is 
unstructured, amorphous, and perhaps even chaotic. 

The processes that a researcher performs at the prehypoth¬ 
esis stage defy generalization and the stereotyped notion of the 
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i( M archer’s moving unerringly from a vague concept to a 
irliued and validated theory is largely a myth. In fact, it may 
i <>mc as a surprise to the new researcher that the most agonizing 
phase of a research activity is the pre-hypothesis stage. Having 
lor'iiiulated testable hypotheses, the avenue of progression for 
the researcher is clear. He can delineate what must be 
accomplished, he can specify the sequence of events, he can 
predict the form of his final product, and with experience, he 
( an probably tell how long it will take to complete his inquiry. 
But it is indeed the rare individual who can anticipate what 
conditions and how much time will be required to formulate 
hypothesis. 

Indu ction 

Induction is the intellectual process of drawing generaliza¬ 
tions on the bases of observations or other evidence. Induction 
lakes place when one learns from experience. For example, 
induction is the process by which a person learns to associate 
the color red with heat and heat with pain, and to generalize 
these associations to new situations. Obviously, induction is 
essential not only for the transmission of knowledge, but also 
for survival. 

Induction is a process of discovery. Induction occurs when 
an analyst or researcher begins to see certain relationships in the 
phenomena he is observing. For example, an analyst or 
researcher might notice from his systematic examination of 
foreign Broadcast Information Service (FBIS) reports that 
unusually bellicose statements were uttered by spokesmen of 
( ountry “Z” prior to the time when arms agreements 
concluded with Country “Y” were announced formally. Or the 
analyst may have noted that an invariant sequence of events 
preceded Country “Z’s” nuclear tests. 

Induction occurs when one is able to postulate causal 
relationships. Intelligence estimates are largely the result of 
inductive processes, and, of course, induction takes place in the 
formulation of every hypothesis. Unlike other types of intellec- 
tual activities such as deductive logic and mathematics, how¬ 
ever, there are no established rules for induction. 
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Describing how induction takes place is tantamount to 
describing how one conceives ideas. Obviously, knowledge and 
experience are important for one to generate ideas, and certain 
personal traits of the investigator, such as curiosity, or an 
obsession with a lack of closure, are important contributors to 
induction. Imagination and “powers” of observation clearly 
figure into the inductive process. 

Physical and social environments can also be created that 
are conducive to induction—environments in which a researcher 
may try out ideas without fear of ridicule from his colleagues, 
for example—but different people react differently to various 
conditions, and for some researchers an element of stress, 
anxiety, or discomfort is conducive to sharpening creative 
powers. 

Observations usually precede all valid generalizations. Al¬ 
though one cannot guarantee that making numerous observa¬ 
tions will necessarily yield generalizations, one can state that 
without observations, the likelihood of developing valid general¬ 
izations is indeed remote. 

The objective of the inductive phase in the hypo- 
thetico-deductive process is to arrive at a number of testable 
hypotheses. Since the quality of the hypotheses is a function of 
the knowledge and experience of the observer, one way of 
improving hypotheses is to involve several people in the 
hypothesis formulation (inductive) phase. At times it may also 
be profitable to employ a “naive observer”—one who is not 
deeply involved in the problem at hand. Naive observers 
contribute by questioning basic assumptions, assumptions 
which those deeply involved in an activity may take for granted. 
The inductive phase of the hypothetico-deductive process ends 
when testable hypotheses are formulated. 

Deduction 

Deduction is the process of reasoning from general rules to 
particular cases. Deduction may also involve drawing out , or 
analyzing the premises to form a conclusion. 2 

2 The word “deduction” comes from the Latin deducere which means 
to draw out. 
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In the example of induction mentioned earlier, an analyst 
noted a pattern of events relating to Country “Z’s” nuclear 
tests. Ilis generalization at that point may have taken the form: 

I vents 1,. . . n, always precede nuclear tests in Country “Z”. 
Alter arriving at this generalization, the analyst may have 
received reports that Events 1,. . . n were occurring in Country 
"/”. On the basis of this information, the analyst would 
conclude that Country “Z” was about to test another nuclear 
device. Initially, the analyst reasoned inductively from numer¬ 
ous observations to a generalization. When the analyst reasoned 
I mm a generalization to a specific case, he reasoned deduc¬ 
tively. His reasoning progressed from premises to a conclusion. 

I he major premise was that Events 1,. . . n always preceded 
nuclear tests in Country “Z”. The minor premise stated that 
| vents l,...n had been reported. Assuming that Events 
I, . . . n were not associated with any other activity, the logical 
conclusion that would be reached was that Country “Z” was 
about to test a nuclear device. 

The example just cited was overly simplified, and like most 
simplifications, it glossed over points that might make a 
significant difference in intelligence research. 

Deduction works best in closed systems such as mathe¬ 
matics, formal logic, or in certain kinds of games-war games 
played on a computer, for example, in which all the rules for 
playing the game are clearly spelled out. In the statement: 
"This is a triangle; therefore the sum of the interior angles will 
equal 90 degrees,” the validity and truthfulness of the conclu¬ 
sions would be apparent to anyone with a knowledge of 
geometry. Conclusions, if drawn correctly in closed systems, are 
always valid. 

But intelligence research rarely deals with closed systems; 

Iluis,conclusions may still be drawn correctly, but the premises 
from which they are drawn may not be true. In the example of 
the nuclear tests, for instance, Country “Z” may have assumed 
that its activities were being monitored, and, for strategic 
reasons, embarked on a large-scale deception program. In this 
case, the premise that Events 1, . . . n always precede nuclear 
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tests in Country “Z” would be false, and consequently, even if 
the researcher or analyst reasoned validly from the premises, the 
conclusions would still be false. The implication for intelligence 
research is apparent: human activities rarely involve “closed 
systems” in which conclusions necessarily follow certain prem¬ 
ises. Therefore, in intelligence research, deduction must be 
used carefully with a full awareness of the limitations of the 
processes, and with an awareness of potential errors in the 
premises.3 

Induction and Deduction in 

the Hypothetico-Deductive 

Process 

It was mentioned previously that in the hypothetico - 
deductive process, induction ended with the formulation of 
testable hypotheses. Hypotheses, it will be recalled, are theories 
or unproved assumptions which relate or explain different facts. 
The purpose of an hypothesis is to guide the researcher in his 
search for evidence. Without hypotheses to delimit the boun¬ 
daries of the search, the researcher would be unable to ensure 
that any collected data were relevant or germane to his 
purposes. As Darwin pointed out, “ . . . all observation must be 
for or against some point of view [an hypothesis], if it is to be 
of any service.” 

Again, the greater the number of hypotheses formulated, 
the greater the chance that one of the hypotheses will prove to 
be the correct one. This is important to remember because there 
is a tendency among many researchers to “lock in” early on a 
favorite hypothesis to the exclusion of others. The number of 
hypotheses that might be generated is infinite, but at the 
minimum, the researcher should formulate at least three. * * * 4 

^Rules for deduction can be found in most textbooks of logic. 

Examples of deduction and induction applied to intelligence problems are 

contained in the series of programmed texts, Logic for Intelligence 
Analysts, mentioned earlier. 

4 E.g., the original hypothesis, the null hypothesis (which is essentially 
the refutation of the original hypothesis), and one more hypothesis that is 
completely independent of the other two. 
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Typically, hypotheses in problem solving contexts are stated 
in the form of the syllogism, i.e.: “If , then .” This 
I us! statement constitutes the major premise, or the generaliza- 
Hon. The next statement (the minor premise) would include the 
m iual evidence to date. To return to the nuclear test example, 
the syllogism would look like this: 

If Events 1, . . . n, then a nuclear device will be tested. 

(Major Premise) 

Events 1,. . . n, have been reported in Country “Z”. 

(Minor Premise) 

Therefore, a nuclear test is imminent in Country “Z”. 

(Conclusion) 

The relationship of inductive and deductive reasoning to the 
hypothetico-deductive process (the scientific method) is shown 
in f igure VIII-1. 

As critical as these two reasoning processes may be, it 
should not be assumed that every research program will 
necessarily require that both processes be performed. Nor must 
the researcher formulate hypotheses for every type of intelli¬ 
gence research project. Certain types of intelligence problems 
will require only the addition of new information. In other 
cases, the hypotheses may be given to the researcher for testing 
as part of his assignment, thus eliminating the need for the 
researcher to draw initial generalizations. Sometimes a research 
project will involve only the inductive phase and the product 
will consist of a generalization that was not yet tested.However, 
in the case of self-initiated intelligence research efforts, it is 
difficult to see how one could progress systematically and not 
employ the hypothetico- deductive process. 
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Figure VIII-1. Relationship of Inductive and Deductive Reasoning to the Hypothetico-Deductive Method 
(These processes can also be considered in terms of discovery and verification as will be 
discussed in Chapter XIV.) ’ 


Summary 


t>The terms hypothetico-deductive method and scientific 
method mean the same thing; however, the term hypothetico- 
deductive method is a more accurate description of the process 
and it does not suggest that the method is necessarily limited to 
scientific investigations. 

t> Induction is the intellectual process by which one arrives at 
generalizations on the bases of observations or other evidence. 
Induction is the process by which testable hypotheses are 
formulated. 

t> Deduction is the process of reasoning from general rules to 
particular cases. Conclusions drawn deductively (and correctly) 
in “closed systems” such as mathematics and formal logic are 
always valid. Unfortunately, rarely does intelligence research 
involve closed systems. 

t>Induction is involved in the process of discovery, e.g., 
noting certain relationships among phenomena that previously 
were not known to exist. Deduction is involved in testing and 
verifying the generalizations formed inductively. 

>Not all research requires both induction and deduction. 
Sometimes a research project (assigned or self-initiated) may 
conclude with a set of generalizations. Other times a research 
program may be initiated to test a set of generalizations. 
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CHAPTER IX 

PLANNING THE RESEARCH PROGRAM - 
PART ONE: PROBLEM DEFINITION 


The adolescent envisions the researcher as one who 
dreams up creative ideas, the needed resources miracu¬ 
lously appear , and the hero, in a state of eager anticipa¬ 
tion, begins his investigation. Those of us who are 
researchers would add some less romantic steps, one of 
which, the preparation of the proposal to make the 
resources “miraculously appear ”, has become quite im¬ 
portant. On occasion, it can be a serious hurdle to the 
implementation or trial of an important idea. 

David R. Krathwohl 

Every well-performed research project involves four major 
phases: problem definition, data collection, data analysis, and 
report preparation. The normal sequence of events progresses 
from the translation of an identified need into a plan of action, 
the execution of that plan of action, and the submission of the 
results of the planned research activities. 

There is nothing sacrosanct about the sequence in which the 
steps are performed. In many instances in intelligence research, 
data collection may occur before a problem is defined or 
becomes apparent. Very often data analysis begins before all of 
the data are collected, and report preparation usually begins 
before all of the data are analyzed. 

This chapter discusses the steps that should take place in the 
problem definition phase of a research project. This is a critical 
phase of the project because decisions made at this phase affect 
the manner in which the entire project will be conducted. 

At the outset, it should be noted that the problem 
definition phase involves more than merely selecting a research 
topic. The problem definition phase is, in reality, a planning 
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phase in which all of the factors which would influence the 
successful completion of a proposal effort must be considered. 

The problem definition phase involves two major functions: 

I) defining the problem, and 2) determining the feasibility of 
the purposed approach for attacking the problem. These 
functions are interrelated. The nature of the problem deter¬ 
mines what resources are required by the researcher. The 
available resources, time, funds, and manpower determine 
whether or not the problem as stated can be addressed 
effectively. 

Problem definition is an iterative process of selection and 
refinement. On the basis of need and the capabilities and 
interests of the researcher, general problem areas are identified. 
Within these larger problem areas, a number of tentative topics 
may be chosen. These topics, in turn, are examined initially 
with respect to their value to the intelligence community and to 
the researcher, and then with respect to the feasibility of 
actually conducting the research. 

Defining the Problem 

The objective of this step is the identification of a research 
topic. This section will discuss the factors that should be taken 
into account as the research problem is defined and the topic 
selected. The factors include the sources and origins of problem 
areas, the need, and the many decisions that must be made in 
delineating the scope of the proposed research project, such as 
specifying terms of reference and operational definitions. 

Sources and origins of 
research projects 

Research projects evolve from recurring needs of a spon¬ 
soring agency or headquarters, from specific non-recurring 
needs of an agency or headquarters, or from problem areas 
perceived by the researcher himself. Examples of research 
projects that are assigned are periodic reports and updates. 
Usually these types of research projects are scheduled and 
generally pose no special problems to the researcher. Another 
type of research project that is assigned are special ad hoc 
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studies. This type of project might cause problems because 
deadlines are often short, the issues are critical, and data may 
not be available. An advantage of all assigned projects, however, 
is that the terms of reference are spelled out in the assignment 
Thus, the researcher need not ponder the scope, length, 
complexity, operational definitions, availability of funds, need,' 
nor the feasibility of the project as he must do with 
self-initiated projects. 

Self-initiated research projects might evolve from a problem 
an intelligence staff officer encountered on his last tour of duty 
or from chronic problems facing the G-2 or J-2 Section of the 
organization to which he will be assigned. Perennial problems 
facing the intelligence community are always potential sources 
for research topics. These problem areas can be spelled out in 
detail in discussions with various agency spokesmen. Conceiva¬ 
bly, a research project currently underway at an agency might 
be enhanced by a supplementary effort, provided that the 
supplementary effort could be completed in time to be of value. 
Current intelligence reports, and even collection requirements 
or Statements of Intelligence Interest (SITs) might suggest 
topics for self-initiated research programs. 

Need 

A proposed self-initiated project that addresses a critical 
need stands an excellent chance of being approved and funded. 
Projects that address peripheral needs or potential future needs 
require that the researcher explain where and how the results of 
the effort would be of value to the intelligence community. 

At least part of this justification would include ascertaining 
whether or not the study has already been done. This should 
entail a search of the various agencies’ classified holdings as well 
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•is an initial survey of the open literature. 1 This initial literature 
survey also helps the researcher determine whether or not there 
is sufficient information available to conduct the study. 

Simply because a study had been done in the past does not 
mean that the researcher must rule out the possibility of 
repeating the study or carrying out a similar research project. 
Certain types of subject matter have short “shelf life.” An 
example would be a general population’s attitudes toward a 
specific issue. As conditions change, public attitudes may 
change accordingly and, therefore,may warrant reexamination. 
Personal, military, and political alignments change, sometimes 
m surprisingly short time, and findings produced by an earlier 
study may be sufficiently obsolescent five years later to warrant 
reconsideration. 

Studies in which methodological problems hampered data 
collection or analysis are always logical candidates for reexami¬ 
nation when new techniques become available or when military 
or political conditions change so that the researchers have access 
to areas that were formerly denied. Sometimes the discovery of 
new information such as diaries, memoirs, or private correspon¬ 
dence may cause earlier findings to be reexamined. The same is 
true when formerly highly confidential official documents are 
made public. 

Experiments in which the original researcher failed to 
consider certain factors, or experiments in which the then- 
existing technology did not permit accurate observation and 
measurement are candidates for reexamination, as are studies in 
which bias on the part of the original researcher is suspected. In 
short, because a study was done in the past does not rule out 

*The researcher might want to examine Dissertation Abstracts (Ann 
Arbor, Michigan: University Microfilms, Inc.) at this step. Dissertation 
Abstracts is a monthly compilation of doctoral dissertations submitted by 
over a hundred institutions. Various professional organizations also publish 
abstracts of research reports or descriptions of research in progress. 
Abstracts of studies completed at the services’ senior colleges might also be 
examined, for example, the Journal of Abstracts published by the Naval 
War College. 
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the possibility of doing it again. The criterion to consider in 
repeating earlier studies is the likelihood of producing new 
knowledge of value to the intelligence community. 

Scope 

Limiting the scope of the research project is one of the 
more critical early tasks the researcher must perform. Typically, 
the researcher underestimates the magnitude of his proposed 
effort. For example, he may not be aware of the vast quantity 
of data he must screen in order to obtain only a few kernels of 
useful information. Furthermore, unless a topic is delineated 
carefully, the researcher may find that his topic has no realistic 
bounds. For instance, a researcher examining the causes of the 
Bolshevik Revolution in Russia could very easily progress 
backward through the decline of the Romanovs, to the decline 
of the Roman Empire, to the rise of Constantinople, and to the 
birth of Christ! 

An intelligence researcher with an interest in history and 
geography might select initially “The Strategic Significance of 
the Dardanelles” as a research topic only to discover that 
without addressing the who and when questions, adequate 
coverage of the topic would require at least some consideration 
of the Graeco-Persian wars and of all intervening conflicts since 
the fifth century. Interesting, perhaps, but this background 
coverage would probably be of little value today. 

The cast of characters expands exponentially in historical 
research, and without limits or guidelines for determining who 
will be studied, the researcher may be soon overwhelmed by the 
comprehensiveness and magnitude of his poorly defined topic. 

Terms of reference 

Delineating the scope involves establishing terms of refer¬ 
ence. Terms of reference are descriptors which define the 
boundaries of the research effort. Terms of reference can be 
established with respect to time periods considered, with 
respect to specific geographic locations, with respect to events 
or movements, and with respect to significant personalities to 
be considered in the study. 
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Defining terms of reference also includes: 

Stating the problem 

Stating assumptions relating to the subject matter 
Stating the hypotheses if the nature of the research is 
such that hypotheses must be tested 2 
Formulating operational definitions 
Proposing a title (which may contain many of the same 
terms used in the statement of the problem) 3 
Describing briefly the methodology to be used (either in 
data collection or data analysis) 

Describing sources of data if the sources are unusual 
Indicating completion dates (anticipated dates, if the 
project is self-initiated) 

Estimating the manhours required (if the project is 
assigned) 

Identifying coordination required and any additional 
assistance needed from support groups (photo¬ 
interpreters, or translators, for example). 

In the case of an assigned project, or in the case of a project 
that is essentially a replication of an earlier project, an initial 
outline of the anticipated final product might also be submitted 
as a term of reference. An outline makes explicit to those who 


2 Not all research required hypotheses. For example, a survey to 
determine the status quo, or to establish what exists at present, would not 
necessarily involve hypotheses. 

3 Writing titles is an art. Intelligence reports, unlike literary works 
produced for public consumption, do not (or should not) depend upon a 
provocative title to attract readers. Numerous problems arise when a title 
does not accurately reflect the content of a report. The amount of time 
and money wasted in cataloging, retrieving, and examining documents 
whose titles are misleading is mind-boggling. If possible, the title should 
contain the what, where, when, or who, if important individuals are 
discussed in the report. In the process of delineating the problem, the title 
may be revised several times before a research project is completed. 
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must approve a proposed effort the breadth of coverage and the 
extent of detail that the researcher proposes to provide. In the 
case of self-initiated projects, however, the researcher will more 
than likely be forced to revise the outline constantly as his 
project progresses, hits unanticipated snags, and employs 
alternative collection or analysis procedures. 

The nature of the outline also depends upon the number of 
researchers working on the effort and the experience and 
personal preferences of the researcher. Generally, efforts in¬ 
volving the contributions of several researchers require a more 
structured outline than would an effort staffed by one person. 

Then, too, there are individual preferences. Some re¬ 
searchers prefer to synthesize a highly detailed skeleton of the 
report initially and then “flesh out” the skeleton with specific 
information and transitions. Other researchers prefer to work 
with only a very general outline which is refined and revised as 
data are collected and analyzed, and as implications of the 
analyzed data become evident. 

Certainly the experience of the researcher is a factor to 
consider in deciding how detailed an outline should be. 
Experienced newspaper reporters,' for example, can write a 
story quickly without preparing a detailed outline simply by 
stating the most important fact first, followed by the next most 
important fact, and so on. The outline is a tool for preparing a 
report: it is not the report itself. Consequently, the amount of 
time spent preparing the outline should be proportional to the 
scope of the overall effort. 

Operational definitions 

Operational definitions must be made at this stage. An 
operational definition is the writer’s translation of a concept 
into observable (empirical) terms. To put it another way, an 
operational definition is a concept stated in terms of that which 
can be observed and,ideally, in terms of the means by which the 
concept can be measured or quantified. 

The concept of “net capabilities” is an example of a term 
requiring operational definition. With respect to net capabilities, 
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tlu- questions must be raised: To what specific action do the net 
capabilities relate, and in what units are the capabilities to be 
expressed? “Power”, “revolt”, and “instability” are also com¬ 
mon terms that require operational definitions. 

There need not be universal agreement on the operational 
definition (although it helps), but the writer must make clear 
what he means when he uses abstract terms such as leadership, 
power structure, revolt, strategic significance, and so on. Again, 
lliis process of spelling out operational definitions also helps 
delineate the scope and establishes the referents by whic 
conclusions can be arrived at and evaluated. 

liroad topics or limited topics? 

Delineating the scope of the effort requires the researcher to 
decide if he intends to treat a broad topic in general terms or a 
limited topic in precise terms. It is the broad topic that causes 
problems. Typically, the scope of a broad topic is so great that 
it exceeds the constraints of cost, time, and manpower. 

Another problem with broad topics is that often the 
treatment tends to be superficial and, thus, of little value to 
anyone who needs substantive information. To produce useful, 
substantive information about a broad topic area, the researc er 
must know his topic well before he begins his project, and this 
criterion excludes most students. 

This is not to rule out categorically any consideration of 
broad topic areas. At times coverage of a wide topic area can 
provide very useful information. Examples of broad topic 
coverage might be evaluations of previous research, or surveys 
of literature relating to a specific topic. Generally, broad topic 
coverage is appropriate in new areas or in areas in which little is 
known The use to which the product will be put would also be 
a factor in determining whether the researcher should opt for a 
broad coverage or a detailed specific coverage,, 

4p)egrees of instability, for example, can be expressed in terms of 
events corresponding to points on a Guttman Scale-events such as the 
resignation of cabinet officials (at one end of the scale), and a coup d etat 
and civil war at the other end of the scale. 
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Determining the Feasibility 

of the Approach 

The objective of this step in the problem definition phase is 
to determine whether or not the topic can be addressed in lighl 
of constraints of time, manpower, available resources (including 
information), and the capabilities and interests of the re¬ 
searcher. 

Time 

The constraint that permeates every type of intelligence 
research is time. Even the student in the academic environment 
of the Defense Intelligence School has time limitations in which 
his research must be completed. Researchers in an operational 
environment typically have shorter response times, but these 
research projects are often team efforts, and in many cases the 
addition of other researchers can compensate for limited time. 
Nevertheless, the length of time available is a major determinant 
in the scope of a proposed project, and this constraint makes 
realistic scheduling and planning all the more critical. 

Again, inexperienced researchers typically underestimate 
the amount of time that will be required to carry out their 
proposed program. Most government-sponsored research neces¬ 
sitates coordination. If the research involves contact with 
foreign nationals, coordination requests may have to be 
transmitted from the U.S. Department of State (or Defense) 
through the hierarchy of another government. The researcher 
may have to spend many hours explaining the nature of his 
request to headquarters of every echelon through which the 
request passes. Protocol is important, chains of command may 
not be bypassed, and all of this takes time. 

Admittedly, access to remote computer terminals may save 
time in collecting archival data, but trips to archives may be 
required nevertheless. Again, procuring travel funds, forwarding 
clearances, establishing points of contact at the visited agency, 
and making protocol visits take up valuable time. Especially 
frustrating are instances in which “leads” fail to yield any 
substantive information, or instances in which archives which 
should contain the required information yield little of value. 
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lllind alleys are encountered in every research project. They are 
exasperating, costly and time-consuming. But they must be 
anticipated. 

Manpower 

At times the addition of other team members can speed up 
research activities, as mentioned earlier. Additional manpower is 
especially helpful in maintaining files, obtaining reference 
material, checking sources, and preparing copy for publication. 
Planning and scheduling are important to ensure that assistance 
is available when it is needed most. 

Available resources 

The amount and quality of resources at the disposal of the 
researcher help determine the scope of the research effort. 
Resources include sources of information—both archival and 
Imman—as well as data collection capabilities, and analysis and 
processing capabilities. With respect to available resources, the 
intelligence researcher has a distinct advantage over his 
academic counterpart. The intelligence researcher with the 
appropriate clearances and the need to know has access to 
virtually every archive maintained or supported by the U.S. 
Government. However, the researcher still needs to know what 
exists, where it exists, and how to go about obtaining what he 
needs. In the unlikely event that the information he needs does 
not exist in any government archive or non-governmental 
repository, and if the research topic is considered sufficiently 
important to warrant it, collection requirements might be levied 
on appropriate field offices and organizations. 

Washington, D.C., and the immediate area surrounding the 
District, probably have a greater concentration and variety of 
data processing equipment than any other place in the world. 
With the remote terminals and time-sharing equipment available 
at the Defense Intelligence School, gaining access to a computer 
should pose no serious problem to the researcher. 

Unlike the problems which face most researchers, the 
intelligence researcher may find that he has too much informa¬ 
tion at his disposal. His job, then, is one of optimizing his data 
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collection and analysis so as not to become overwhelmed by the 
volume of information. Again, established terms of reference 
and (later) hypotheses provide guidelines for determining that 
information which is extremely important, and that informa¬ 
tion which is of peripheral interest. 

In certain types of research programs, as mentioned earlier, 
data already existing in the various repositories (archives, data 
bases, etc.) will not satisfy the researcher’s needs. In these 
instance^ the researcher must plan special data collection 
eftorts. These collection efforts may include making firsthand 
observations (naturalistic or controlled), or administering ques- 
tionnaires, or conducting interviews. 

Even when only archival data are used, it may be necessary 
to devise formats for recording information. Consequently the 
researcher must also consider the manner in which he proposes 

to collect his data in the initial planning and problem definition 
stages. 

Not to be overlooked are methodological problems in 
ana yzing data. When limited amounts of archival data are used 
analysis can be performed without special facilities. However, if 
large quantities of raw data must be reduced, or if complex 
operations must be performed on the data, then the use of data 
processing equipment is essential. Again, this must be antici¬ 
pated in the planning stages. 

Cost 

Self-initiated research efforts which can be completed with 
existing materials, sources and facilities do not pose problems in 
funding. Problems arise, however, when travel is required, when 
special collection activities must be implemented, or when large 
amounts of time must be spent on data processing equipment 
Communications can be expensive when AUTOVON facilities 
cannot be used, or when much written correspondence is 
required. If the research topic is not perceived to be of critical 
value, the researcher may have to cover these costs personally. 

In addition, the researcher may have to cover the costs of 
clerical assistance. The costs of materials should not be 
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ovcrlooked-materials such as basic references, in some in¬ 
stances, and the minutiae required by every research project- 
stationery, cards, typewriters, and so on. Projects that require 
the use of special equipment—laboratory facilities, for 
example—may increase costs. Surveys and polls involve many 
additional costs such as postage, printing of questionnaires or 
interview schedules, and interviewer fees (in some cases). 

Should data collection require travel in the hinterlands of a 
foreign country, special personal equipment may have to be 
procured, and if data collection involves several people, 
obtaining this equipment may be tantamount to outfitting an 
expedition. 

All research activity involves some expense, and considera¬ 
tion of available funds should enter into all research planning. 

Capabilities and interests 
of the researcher 

The researcher must not overlook his personal knowledge, 
skills, and interests when he selects a research topic. For 
example, a researcher with no knowledge of research design or 
of basic statistics would be well advised to stay away from 
programs involving laboratory experimentation. Researchers 
who require extensive use of data processing equipment need 
certain skills and knowledge if they are to use these facilities 
effectively. Admittedly, research is a learning process, but 
without a minimal level of background knowledge or skills, the 
researcher is in a poor position to make even a modest 
contribution. In an academic environment, the researcher may 
have to take time to develop requisite skills or acquire sufficient 
background in a subject area in order to address it effectively; 
but in light of the time constraints in intelligence, this is rarely 
feasible in an operational environment. 

Because of dispositions, personal proclivities, and so on, 
certain types of research activities or subject matter may be 
distasteful to the researcher and nothing can be more stultifying 
than poring over screed that is tedious, monotonous, and 
uninteresting. Invariably, the research product reflects the 
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enthusiasm and interests of the researcher; therefore selectin 

IZr S " ° ne me,h0d of ^intainta’/tuS; 

Feasibilit y: The All-Inclusive 
Issue 

Throughout all of the initial planning steps of a project nm- 
question must be asked constantly: Is the proposed 1'fSr 
easible. Limiting the scope, ensuring the availability of funds 
and planning for sufficient lead time increase the likelihoodThai 

the program can be completed successfully. But these facto ' 
cannot guarantee success. iactors 

rion L in elih ° 0d ° f successful completion is an important crite¬ 
rion in gaming approval for a self-initiated project If anv 

fundlng ls inv olved in the self-initiated project 
ntelligence managers (and faculty advisers) want some reason’ 
able assurance that the effort will be worth the exnenihw 
ven if funding is not involved, the researcher is making a large 

Imhcart'-horr 5 °T ‘"" e r nd ener8y ' Eveiy graduate student 
for years until mmovtd 

burdensomrS 1 fhe are 

intelligence community nonproductive to the 

qv .. A “ de f rom 3 to P ic that may be too broad with respect to 
available time and resources, factors that contribute most to 

TTi a erefore P the eCt Unfeasible are nonexistent or unavailable data 
therefore, the researcher should pay especially close attention 

need P s n "E Tt ^ “"-<ing“he d“a h 

needs. That data relating to a specific topic do not exist in anv 

archive or in a form in which they can be used is noting new 
Dortion S ‘f Why data . collection typically takes up such a large 

She n rp 0 rob,em SearCh ^ ^ ' ha ' ^ 

Data may be unavailable for a number of reasons- there mav 

the sL a e ny of Se t n he 0r aT ble ? de \ Cting 3 Certai " emanatL or 

rne state of the art may be such that a sufficient degree of 
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resolution has not been attained to make a detected emanation 
useful. But these are secondary conditions relating to a larger 
problem, namely, inaccessibility to the target of interest. A 
target of interest may be a fortification, a production facility, 
or a weapons system. But the target of interest may also be a 
sample of the opposition’s general population, or the party 
cadre in a capitol city. In sensitive political environments, even 
friendly populations may be inaccessible to the researcher, and 
(lie likelihood of his acquiring data directly from these 
“denied” groups is very remote. Problems in obtaining informa¬ 
tion about “denied” groups may be so great as to warrant 
research in their own right.^ 

In other cases, data might be obtainable, but only at great 
risk to the collectors. Here a judgment must be made as to 
whether the data are worth the risk entailed in collecting them. 
It is highly unlikely that any self-initiated program involving 
risks to lives or equipment would be approved. If the topic 
addressed were indeed so critical, the project probably would 
have been assigned in the first place. 

The researcher should consider that even when he has direct 
access to the data, political or military conditions may change 
suddenly and the data may become inaccessible. One method of 
increasing the feasibility of the proposed effort is to plan a 
series of alternative data collection procedures. Ideally, the data 
collection procedures would range (initially) from those in 
which the researcher could make direct observations, to those in 
which the researcher could make only inferential judgments. As 
undesirable as inferences may be compared to direct, firsthand 
observations, making decisions on inferences when facts are not 
available is not uncommon in intelligence. 


5 A case in point would be the studies funded by the Department of 
Army to devise techniques for assessing the effectiveness of psychological 
operations directed toward denied audiences. 
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Problem Definition: The First 
but Not Final Phase of the 
Planning Activity 

It should be apparent that the process of defining tin 
research problem involves more than simply picking a topi 
Defining the problem involves consideration of every phase < 
the entire research program. Defining the problem is, in fact, 
micro research program in its own right. Factors that determini 
whether a proposed program is relevant and feasible art, 
interrelated, and any one factor in a self-initiated program 
might impact sufficiently on the others as to make the entire 
program unfeasible. 

Defining the problem is the first phase in the planning 
process. But this phase does not end when the next phase 
begins. Throughout the conduct of the overall project, the 
original problem may be further refined and delineated. It is not 
unusual to discover in the course of a research program that 
certain questions subsumed under the larger original question 
were more important than the initial topic. The distillation 
process is common (and desirable) in self-initiated research 
programs, and the more specific a topic becomes, the greater 
the likelihood that the researcher will be able to conclude his 
project successfully. Steps in the problem definition phase are 
shown in Figure IX-1. 
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Figure IX-1. Steps in the Problem Definition Phase 



























































Summary 


t>Mos t research projects involve four phases: problem defini- 
on, data collection, data analysis, and report preparation 

Planning is essential in all phases. p 

> The problem definition phase of a research project involves 
two major functions: defining the problem and determining the 

problem^ ° * approach that is Proposed for attacking the 

0 Defining the problem involves considering the need for the 
study, delimiting the scope, defining terms of reference, and 
stating operational definitions. 

C> Determining feasibility requires the researcher to consider 
the time he has available to complete his project, available 
manpower, available resources (such as sources of information), 
cost, as well as his personal capabilities and interests. These 

JSEL™ lnt ® rrelated -. For example, data not immediately 
available may be expensive to acquire, take time, and require 
additional manpower. Additional manpower, in turn may 
increase costs. ’ y 

>In self-initiated research projects, the researcher may further 
re me his topic throughout the problem definition, data 
collection, and analysis phases of his project. 

> The ultimate criterion in selecting a research topic is need 
The ultimate criterion in determining if and how a research 
topic should be addressed is feasibility. 
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CHAPTER X 

PLANNING THE RESEARCH PROGRAM-PART TWO: 
LOCATING SOURCES OF INFORMATION 


Knowledge is of two kinds: we know a subject ourselves, 
or we know where we can find information upon it. 

Samuel Johnson 

In order to determine the nature and extent of the data 
collection activities to be performed, the researcher must know 
what data exist, and where and how data can be obtained. This 
chapter will discuss characteristics of various types of data, and 
places where data can be obtained. 

Primary and Secondary Sources 

Primary sources of information are firsthand observers, 
participants, or recorders. For example, a participant in a 
military operation would be a primary source, and any account 
he prepared pertaining to the operation, such as an after action 
report, would be a primary source document. An attache who 
observed a training exercise would be a primary source; so 
would an agent or a defector who made firsthand observations 
be a primary source. The outputs of various sensors (ELINT 
receivers, other receivers, radars, cameras, and scanners) are 
primary source data. 

Secondary sources are documents prepared on the basis of 
primary sources. Thus, if an analyst prepared a report based 
upon evaluated information derived from interviews with agents 
or defectors, his document would be considered a secondary 
source. 

Many researchers assume that primary sources or primary 
source documents are necessarily more reliable than secondary 
sources. This assumption is unwarranted. In evaluating any 
information, both the competency and the motivation of the 
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source must be taken into account. In certain instances 
observers are not competent to report what they observed. Foi 
example, an untrained observer might report an armored car as 
a tank, or a company-sized unit as a battalion. 

Sometimes people see only what they want to see. For 
example, in World War II an enemy fighter plane downed while 
attacking a formation of bombers might be claimed as a “kill” 
by nearly every gunner in the bomber formation. Inflated 
estimates of enemy aircraft involved in a battle and inflated 
reports of the number of kills plagued intelligence organizations 
on both sides during World War II. 

Motives of the writer must be taken into account when 
primary source documents are used. If a writer realizes that his 
personal records will be used by others he may deliberately 
portray himself in a favorable light. Even the most “candid” 
memoirs tend to be self-serving to some extent. 

Researchers using statements made by prisoners of war must 
pay especially close attention to the motives of the source. 
Prisoners of war may lie or exaggerate in order to appease their 
captors, or to justify their behavior, or to deceive. During the 
Korean War, for example, Chinese Communist troops would 
permit themselves to be captured in order to organize other 
PW’s detained in United Nations prison compounds. And it was 
not unusual for North Vietnamese Army (NVA) prisoners in 
Vietnam to report the favorable reception of U.S.-produced 
PSYOP leaflets in areas where none had been dropped. Rarely 
do prisoners give fear as a reason for surrendering, but instead 
cite ideological reasons, or lack of resources with which to fight 
as reasons for their capture. 

People under stress are often poor observers or reporters 
(except of their own emotions, perhaps), and memories 
sometimes fail in recalling significant details. General S.L.A. 
Marshall found that descriptions by individual participants in 
battle were notoriously unreliable and that only by collating the 
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accounts of many participants could a valid picture of what 
actually occurred be constructed. 

Furthermore, cultural^ tmits mayimpair ^ness of 

primary sources. For exa P , what he thinks the 

a courteous resp0 " d h d f or this reason it is sometimes 

— r dumber Of s, = * —ate 

wrd° primary Source ^ document are no, necessarily the best 
sources of reliable information. 

Intentional and Un intentional 

Transmitters of Facts 

irHSSS 

not the medium y conveyor of information. 

^1—' “ransmitters of ftrctswould **£?££* 

mgTtbis' SToVTansmitters Of historical facts has a 
corollary in intelligence. (Table X-l.) 

The listing of intelligence transmitters of facts makes ^tclear 

that the researcher who limits his data omy f 


iThe Late Cornelius Ryan, author of The Longest Day The Last 

r:,r -s . Y .— 8 

^.ic^nts in Omar to reconstruct an accurate account of wha, happened 
at Normandy, Berlin, and Arnhem, respectively. 

W The Modern Research PP. 

Craft classify letters, public dommen s ^d purpose for 

nreditated —“ ,17w 0 uW be Ponced would be to record fo, 
" H —ft: Which tootace « «- dm. the record was 

made. 
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— Table X-l. Premeditated and Unpremeditated Transmitters 

oo of Facts 


HISTORIGRAPHICAL TRANSMITTERS OF FACTS (BARZUN AND GRAF) 

INTELLIGENCE TRANSMITTERS OF FACTS 

RECORDS 

(INTENTIONAL TRANSMITTERS OF FACTS) 

1 CHRONICLES, ANNALS, BIOGRAPHIES. GENEALOGIES 

2 MEMOIRS, DIARIES 

3 CERTAIN KINDS OF INSCRIPTIONS 

ORAL 

4 BALLADS, ANECDOTES, TALES, SAGAS 

5 PHONOGRAPHS AND TAPE RECORDINGS 

WORKS OF ART 

6. PORTRAITS, HISTORICAL PAINTINGS, SCENIC SCULPTURE, COINS AND MEDALS 

7 CERTAIN KINDS OF FILMS, KINESCOPE, ETC 

RELICS 

(UNPREMEDITATED TRANSMITTERS OF FACTS) 

8. HUMAN REMAINS, LETTERS, LITERATURE, PUBLIC DOCUMENTS, BUSINESS RECORDS 

9 LANGUAGE. CUSTOMS, AND INSTITUTIONS 

10 TOOLS AND OTHER ARTIFACTS 

RECORDS 

(INTENTIONAL TRANSMITTERS OF FACTS) 

1 OFFICIAL STATE HISTORIES, BIOGRAPHIES, GENEALOGIES (eg . THE OFFICIAL 
BIOGRAPHY OF KIM 11-SONG, HO CHI MINH, AND MAO TSE-TUNG; THE REVISED 
BIOGRAPHY OF STALIN ) 

2 KHRUSHCHEVS MEMOIRS; THE PENKOVSKY PAPERS; CAPTURED DIARIES OF PRISON¬ 
ERS; NON-FICTION NOVELS e g . THE GULAG ARCHIPELAGO 

3 FACTORY MARKINGS ON MILITARY EQUIPMENT; INSIGNIA; PRINTED SLOGANS. 

ORAL 

4 PATRIOTIC SONGS; PATRIOTIC SLOGANS; CHEERS; MOTiOES 

5 RECORDINGS OF OPEN PROCEEDINGS OF CONGRESSES;SPEECHES OF THE LEADER; 
TRANSCRIPTIONS OF PRESS CONFERENCES 

WORKS OF ART 

6 STATE-COMMISSIONED DRAMAS, DANCES, MUSClAL COMPOSITIONS; PAINTINGS AND 
STATUARY; NATIONAL SHRINES; TOMBS OF GREAT LEADERS, eg., LENIN’S TOMB. 

7. STATE PRODUCED RADIO AND TELEVISION DRAMATIZATIONS; PROPAGANDA FILMS, 

ETC 

RELICS 

(UNPREMEDITATED TRANSMITTERS OF FACTS)* 

B INTERCEPTED CORRESPONDENCE AND OTHER COMMUNICATION; CAPTURED PLANS, 

ORDERS AND REPORTS; BILLS OF SALE. SHIPPING DOCUMENTS: PRODUCTION 
FIGURES; BUDGETS; ORGANIZATION CHARTS. 

9 CREATION OF NEW WORDS (TECHNICAL VOCABULARY); CAREFUL AND SELECTED 

USE OF DIPLOMATIC LANGUAGE;ATTEMPTS TO EXPUNGE THE WORKS OF EARLIER 
SPOKESMEN, 

10 CAPTURED WEAPONARY, UNIFORMS AND EQUIPMENT 

* TO THE LIST OF UNPREMEDITATED TRANSMITTERS OF INTELLIGENCE FACTS SHOULD BE ADDED RADIATIONS AND EMISSIONS; eg , LIGHT, ELECTROMAGNETIC RADIATIONS, 

HEAT (IR), PARTICLES, ACOUSTIC, AND SEISMIC. 
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But researchers often fail to take into account the bias thev 
may introduce themselves. As mentioned earlier? inexperienS 
researchers categorically reject as “mere proZS' Zt 
statement earned over state-controlled broadcasting systems 
For instance, broadcasts from mainland China and froirf North 
Korea have alluded to industrial production problems and to 
instances of internal dissent. And in the caseTf ChTna often 

or'SSESSi ^* Inaam *° in the party 

One of the major sources of researcher-introduced bias 
nvolves the manner m which the researcher selects and analyzes 
. nH d t u 3 ' F ° r ® xample ’ he may limit his input only to one source 

tion H? S m XC h de ° ther P ° tentially valuable collateral informa- 
data or hi be Unsyste f atic in his selection and collection of 
for his da? a V USe a " alySis tedlniques that are inappropriate 

For example, one of the authors noted that propaganda 
analysts were performing statistical analyses of North Korean 

nTaSe a d W Freou the aSSUmptions of the statistics were 

were jKl /? qUency cou nts, means, and standard deviations 
were calculated on data that were not obtained in any 

ystematic manner. The analyses were based on daily takes 
provided by Foreign Broadcast Information Service (FEffS) As 
useful as the FBIS output may have been, in order to perform 

to brobtlined"? YSeS tn f the kinds just motioned, the data had 
to be obtained from the same source, at the same times for a 

specified period. However, in the case just cited, Tata were 

oHected only when it appeared that something worthwhile 

would be obtained. As will be discussed in later chapters e“n 

in?fH tV ® ampllng must be done systematically if any mean¬ 
ingful statistical analyses are to be performed on the data. 


m mss sj 

— 

otherwise utopian socialist society of North Korea. 
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Anticipating the existence of bias permits the researcher to 
take appropriate safeguards, and at the minimum, to treat his 
data with skepticism. Much harder to anticipate are potential 
biases inherent in the researcher himself, particularly cultural 
biases which not only affect the way a researcher may feel, but 
which may also affect the manner in which a researcher actually 
perceives reality. 5 One method of possibly reducing the effects 
of bias is to use a variety of sources as rigorously and 
as systematically as possible. 


Sources of Research Data: 

People, Objects, Emanations, 
and Records 

Sources of research data may be people, objects , emana¬ 
tions, and records. As mentioned earlier, the sources may be 
primary or secondary. Regardless of the nature of the source, all 
data from all types of sources must be transformed ultimately 
into some symbolic form, typically words or numbers. 

People 

Human sources of research data may be subject matter 
specialists, or they may be “information specialists.” Examples 
of subject matter specialists would be analysts at various desks 
in the different government agencies, journalists, scientists and 
engineers (employed by the U.S. Government, or under 
contract), scholars, emigres, defectors, eye witnesses, or partici¬ 
pants (in an event, a movement, an organization, or an 
operation). These specialists can help the researcher by pro¬ 
viding substantive answers to specific questions, and by indi¬ 
cating additional sources of information. These additional 
sources could be other human sources or documentary sources. 

5 Benjamin Lee Whorfs ? “The Relation of Habitual Thought and 
Behavior to Language,”is a masterpiece in this respect. The essay discusses 
the effects of language, one of the more profound manifestations of 
culture, on perception. The essay can be found in Leslie Spier, ed., 
Language, Culture and Personality: Essays in Memory of Edward Sapir 
(Manasha, Wisconsin: Sapir Memorial Fund, 1941). 















Sometimes the human sources of data are also the subject of 
the research. For example, the former KGB agents interviewed 
by John Barron for his book, KGB: The Secret Work of Soviet 
Secret Agents ,6 were both primary sources of information 
about the KGB’s organization and operations, as well as the 
primary subjects of the book. 

Analysts, scholars, or journalists who specialize in certain 
subject matter areas know the literature in their Field and they 
can save the investigator many hours of searching by indicating 
the best or most recent literature relating to the researcher’s 
needs. When time is critical, this preliminary literature screening 
is invaluable. 

But for increasing research efficiency , the information 
specialist excels. Information specialists are those who are 
intimately familiar with the holdings of various repositories and 
who are trained in the techniques of retrieving information. 
Information specialists include those whose tasks include 
procuring, cataloging, and retrieving documents (i.e., “librar¬ 
ians”, in the traditional sense), and those who specialize in 
retrieving data from automated systems. 

These resource personnel can assist the researcher, again, by 
indicating likely sources of information, but equally important, 
by helping the researcher devise efficient search strategies. For 
example, the librarian can identify indexes relating to various 
subjects, and use of these indexes can save the researcher many 
hours of poring through card catalogs. The librarian may also be 
able to identify special holdings, e.g., certain kinds of classified 
information, or private or public collections of works relating to 
a specific topic and again save the researcher time. Data 
processing experts can help the researcher formulate queries for 


^John Barron, KGB: The Secret Work of Soviet Secret Agents (New 
York: Reader’s Digest Press, 1974). 
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mm-hinc processing so that the response is tailored to the 
ii M archer’s specific requirements. 7 

Nol to be overlooked are research assistants. Admittedly, 

K . r arch assistants may not be available for student use, but 
major commands and the larger intelligence organizations may 
have personnel available who are well-trained in gathering 
ilociimentation from various libraries and other repositories. 

( onccivably, if a research project warranted it, and if the proper 
approval had been obtained, the assistance of the Congressional 
Research Service of The Library of Congress might be em¬ 
ployed. 

In essence, the “holdings” of any library are increased 
ignificantly by the ability to obtain materials through inter- 
library loans or by the ability to tap other automated data bases 
I min remote terminals. Again, by using these services and 
lai ilities, the researcher saves time and money and is spared 
inconvenience. 

In addition to procuring documents (including maps and 
charts), providing general research and bibliographic assistance, 
.md alerting researchers to new acquisitions, the D1A Library 
personnel (specifically, personnel from the Translation Branch 
() f the Central Reference Division) can also provide in-house 
translation service in eight languages.^ 

Finally, with their knowledge of the magnitude (or paucity) 
of available data, their familiarity with what has been done in 
the past, and their knowledge of problems encountered in 


7 ; \ DIA analyst queried his system for information relating to 
underground structures. Unfortunately, the “structure” part of the query 
was not made clear, and the machine “dumped” everything relating to 
underground organizations, the French Maquis, guerrilla warfare, uncon¬ 
ventional warfare, and so on, since World War II. 

^French, German, Romanian, Italian, Portuguese, Spanish, and 
Russian. 
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similar or related efforts, information specialists can also 
provide assistance to the researcher in determining the feasi 
bmty of a self-initiated research project. 

Objects 

Objects are more often the subject of research than they are 
a source of research information. However, the characteristics 
ot an object may suggest something about its origin and 
intended use and in this sense an object is at least a source of 
inferences if not an actual transmitter of facts. A change in the 
composition of a piece of personal equipment, for example 
may indicate a shortage of certain materials; a change in quality 
or design sophistication may indicate a lack of sufficiently 
trained production personnel; the absence of certain equipment 
on later models may indicate that problems were encountered 
with the equipment on earlier models; and machine stampings 
indicate the country which produced the item. 

In intelligence research the objects of concern are weapons 
weapon platforms, and equipment—configurations, navigational, 
ogistical, medical, transportation, and personal. Unless the 
intelligence researcher is technically competent to “exploit” a 
piece of equipment himself, he will require assistance from 
other human sources. Where appropriate resource personnel 
could be located would depend upon the service to which the 
piece of equipment would be of primary concern. For example 
the Naval Intelligence Support Center (NISC) at Suitland’ 
Maryland, would be an appropriate place to locate experts in 
foreign naval equipment; the Foreign Science and Technology 
Center (FSTC) at Charlottesville, Virginia, would be an appro- 
pnate place for expert information on army materiel and the 
Foreign Technology Division at Wright Patterson Air Force Base 
in Dayton, Ohio, could provide assistance relating to aircraft 
avionics and electronics. 

For information relating to missiles, the Army’s Missile 
Intelligence Agency (MIA) in Huntsville, Alabama, would be a 
plausible source (depending upon the type of missile); and 
information relating to medical equipment and practices could 
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Im obtained from the Army’s Medical Intelligence and Informa- 
lloii Agency (MilA) in Washington, D.C. 

Equipment specialists in various branches of the Army 
• mild also provide consultative assistance. Locations of possible 
irsource personnel in various technical areas include: Aberdeen 
1’ioving Ground, Aberdeen, Maryland, ordnance; Ft. Lee, 
Virginia, quartermaster materiel; Ft. Belvoir, Virginia, engi¬ 
neering equipment; Ft. Eustis, Virginia, transportation equip¬ 
ment; and Ft. Monmouth, New Jersey, and Ft. Huachuca, 
Arizona, signal communication equipment and electronic equip¬ 
ment. 

Additional expertise in scientific intelligence, electronics, 
mid foreign missiles and space activities can be found within the 
Directorate of Science and Technology of CIA. However, these 
areas tend to be highly sensitive, and appropriate clearances and 
need to know must be established before these human resources 
v an be used. 

Exactly where to go for what type of assistance is difficult 
lo spell out. For example, a fire direction radar on an 
antiaircraft weapon might be considered the purview of the Air 
f orce, whereas a radar used for battlefield surveillance would be 
of primary interest to Army electronics specialists. Furthermore, 
one technical intelligence organization may be concerned only 
with selected aspects of a piece of equipment. For example, the 
physical features of communication equipment would be of 
interest to specialists at Ft. Monmouth, but the signal character¬ 
istics would be examined at Ft. Huachuca. 

Coordination with the appropriate agencies must follow 
established chains of command. Agency liaison and service 
personnel at the Defense Intelligence School can provide 
guidance in coordinating contacts with the various agencies. 
Information for coordinating visits can be obtained directly 
from the agencies or organizations to be visited. 
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Emanations 

Emanations refer to those detectable phenomena given of I 
or radiated by natural or man-made objects. Since virtual!' 
every war-making device gives off certain emanations, the 
detection and classification of these phenomena are critical to the 
intelligence community. Emanations are essentially bias free 
But bias may be introduced when the human attempts to give 
meaning to a specific radiation or emanation. 

Emanations are unintentional and unpremeditated trans¬ 
mitters of facts, and adversaries often go to great pains to 

\u dU i°fu? r ?!'T a ± e them- For exam P le > ir > the early part of 
World War II, the German navy began to sustain very high losses 

in its submarine fleet. Convinced that British airborne radars 
were detecting waves that were re-radiated from the submarines’ 
receivers, the Germans curtailed the use of the receivers. But the 
casualties continued to mount because the British had devel¬ 
oped microwave radars whose signals could not be detected by 
German receivers even had they continued to have been used. 

e fact that British radars operated on frequencies and power 
output unknown to the Germans forced the Germans to 
develop methods by which subs could remain submerged and 
this gave impetus to the development of the “snorkel”, a device 

that permitted submarines to recharge their batteries without 
surfacing.^ 

Data about emanations are obtained from the outputs of 
various sensors, and the type of sensor determines the nature of 
the output. For example, the output may take the form of 
imagery ( photographic”, infrared, or multi-spectral) or mag¬ 
netic tapes. Exploitation of these data requires highly special¬ 
ized skills. Furthermore, before the data can be used by the 


Jerome A. O’Connell, U.S. Navy, “Radar and the U-Boat,” United 
States Naval Institute Proceedings 89 (September, 1963): 53-65. This 
anecdote also illustrates the dangers of faulty assumptions. German 
electronic specialists assumed that since they had not been able to develop 
radars in certain frequencies, neither could the Allies. 
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uiin technical analyst or researcher, they must be converted into 
.mother form, usually verbal or pictorial. 10 Collection and 
.Dialysis of emanations constitute some of the most highly 
i lassified operations of the intelligence community. Access to 
these data requires special clearances. 

Uncords 

Records exist in symbolic and non-symbolic forms. Sym¬ 
bolic forms of records include all verbal reports (written and 
oral, e.g., tape-recorded interviews) and numerical tabulations. 
Non-symbolic forms of data include photographs (without 
annotations) and outputs of other sensors which are usually 
contained on magnetic tape. Until the non-symbolic forms of 
records are converted to some symbolic form, they must be 
considered as “raw”, unevaluated data. Even many of the 
symbolic records at the disposal of the intelligence researcher 
are raw data; for example, the unevaluated intelligence reports. 

Using non-symbolic raw data requires a high degree of 
specialized technical training, and separate organizations have 
been created specifically for exploiting these types of data; for 
example, the National Photographic Interpretation Center 
(NPIC), and the National Security Agency (NSA). Unless the 
researcher has the appropriate training and skills necessary to 
work with raw non-symbolic data, he is advised not to attempt 
to use them. 

Raw symbolic data are a different matter. Use of these data 
requires only the skills that all researchers should have 
developed—skills in establishing the credibility of the source, 
and the plausibility of the contents. 

Evaluated symbolic data constitute the bulk of the materials 
with which most researchers work. These data are stored in 
libraries, “archives”, respositories, and data bases. Where one 


10 lnfrared (and other) imagery may be converted to digital data for 
certain types of analysis, then back into some pictorial form for other 
types of analysis. 
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goes for the data depends upon the data required, as mentioned 
earlier, but the initial point of contact for students of the 
Defense Intelligence School is the School Librarian. 

The Researcher's Essential 
Elements of Information (EEl's) 

Initial contacts with librarians usually take the form of a 
dialog in which the librarian attempts to determine as precisely 
as possible the researcher’s needs. In the process of this 
discussion certain questions are addressed. These questions and 
their implications are discussed below. 

Question 

1. Is the information needed likely to be classified? 

Implication 

Classified data will require the use of Government agencies’ 
10 dings. Consequently, the number of potential reposi¬ 
tories is limited. Determining security classification also 
indicates which sections of an agency’s library should be 
examined; for example, the unclassified, classified, or 
special sections of the Defense Intelligence School Library 
The determination of classification also indicates to the 
researcher what security clearances he must hold to gain 
access to these data. s 

Question 

2. What agency or department (university, research facility or 
organization) would be a logical source of the required 
information? 

Implication 

This determination is useful for establishing initial contacts 
with resource personnel, as well as for locating printed 
materials. Library and Reference Facilities in the Area of 
of Colum bia, Mildred Benton, ed., (Washington 
D.C.: The Joint Venture) is very useful for this purpose 
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< Question 

t Is the information required considered to be of a “current” 
nature or of an “historical” nature? 

Implication 

For information pertaining to current or recent events one 
would normally search periodicals, monographs, newspapers 
and current intelligence reports as well as indexes of 
periodic literature; e.g., the Readers’ Guide to Periodical 
Literature and The New York Times Index. For “historical” 
information (events at least six months old), books, basic 
intelligence documents and indexes to these documents 
would also be examined. Normally, “scholarly” books 
require at least a year from concept to publication; 
however, publishers can sometimes produce books in 
surprisingly short times; e.g., the Government Printing 
Office’s publication of the Watergate tape transcriptions in a 
matter of days. The Subject Guide to Books in Print can be 
helpful for locating “historical” information. 

Question 

4. Are there bibliographies or indexes relating to the subject 
matter? 

Implication 

Screening these documents may save time, money, and 
travel. The listings indicate what is available, and sometimes 
where the information is stored. Bibliographies also indicate 
who is contributing to the field. (See item #7.) A valuable 
index is the A ir University Index to Military Periodicals. 

Question 

5. Does the information needed exist in any abstracted form? 

Implication 

The Defense Documentation Center (DDC), at Cameron 
Station, Alexandria, Virginia, has abstracts of thousands of 
documents prepared for official use by government and 
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contactor personnel. Abstracts can also be obtained from 
the National Technical Information Center (NTIC) at 
Springfield Virginia. Abstracts of books, periodicals, tram 
lations, evaluated and unevaluated intelligence reports on 

o nd technical subjects can be obtained via the 
C1KCOL System (Central Information Reference and Con¬ 
trol On-Line) and the CIRC II System. Abstracts of articles 
trom some sixty periodicals as well as from The New York 
Times can be obtained from The New York Times Informa 
tion Bank. 

Question 

6. How soon is the information required? 

Implication 

This information is necessary to make tradeoffs between the 
best sources possible and most immediate sources available. 
Question 

7. Who are the authorities in the fields? 

Implication 

Knowledge of authorities permits data searches on the basis 
ot author. This expedites reviews of literature. Authorities 
are also potential resource personnel as well. 

Question 

8. What are the basic reference works in the field? 

Implication 

Basic reference works exist for nearly every discipline. Many 
° f , ^ese references contain substantive data; for example, 
Who s Who, (and the various international versions thereof)- 
various encyclopedias (of world history, or American 
history); handbooks; yearbooks; and so on. Basic works also 
often contain bibliographies. In addition to basic works 
attention should be paid to the more prestigious journals in 
the respective disciplines. These journals often publish 

indexes annually. Journals also help identify authorities in 
the field. 
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(Question 

Where are the resources located? 

Implication 

Locations of human sources can be determined often from 
the sources in which they publish. Locations of collections 
of published materials can be inferred, in part, by the nature 
of the document (e.g., periodical, book or monograph), by 
the subject matter addressed, and by accession lists and 
catalogs. 

Having identified the kinds of data required and the 
locations or repositories of data, having identified key personnel 
to be consulted, and having assessed the quantity of data 
available, the researcher is ready to conclude the planning phase 
of his research program. The conclusion of the planning phase 
involves specifying the activities to be performed, determining 
the sequence in which they are to be performed, and allocating 
time for each activity. These steps will be discussed in the next 
chapter. 
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Summary 


C> Knowledge of real or potential sources of data is essential 
for project planning purposes. The nature and extent of the 
subsequent data collection effort will depend upon the avail 
ability of sources. 

[> Sources are classified as primary or secondary . A primary 
source would be a firsthand observer, participant, or recorder of 
an event or activity. A secondary source is a document prepared 
on the basis of a primary source. Primary sources are not 
necessarily better than secondary sources, because some first¬ 
hand observers or participants may be poor observers, may be 
poor recorders, and may deliberately report only that which 
tends to reflect most favorably on themselves. 

>Data or information may be classified also on the basis of 
whether or not the medium by which a fact is transmitted was 
created intentionally or unintentionally. Examples of inten¬ 
tional transmitters of facts would be diaries, chronicles, and 
annals. Examples of unintentional transmitters of facts would 
be customs, tools, relics, and artifacts. 

>The concern for possible bias should be taken into account 
when selecting sources of data. Bias may be inherent in the 
data, or may be introduced inadvertently by the researcher. 

> Sources of research data include people, objects, emana¬ 
tions, and records. 

I> The researcher’s tasks in the planning (or problem defini¬ 
tion) phase are to determine what kinds of data he needs, who 
or what are the best sources of these data, and where the data 
are located. 

t> For archival data, the librarian can provide invaluable 
assistance. In addition to actually procuring needed reference 
material, the librarian can also assist the researcher by helping 
him define his data requirements more precisely. 
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CHAPTER XI 

PLANNING THE RESEARCH PROGRAM-PART THREE: 
SCHEDULING THE ACTIVITIES 


Most people spend ninety percent of their research time 
chasing the last, elusive ten percent of the information 
they use. 

Charles Jacobs 

By the time the researcher defines his problem and 
identifies his sources of information, the temptation to start 
data collection immediately is a strong one. The intelligence 
researcher who has “gone through the mill”, however, realizes 
that time spent scheduling the subsequent events will pay off 
well in anticipating technical problems, in anticipating adminis¬ 
trative problems and delays, and in making clear to himself (and 
to his supervisor or adviser) what tasks must be accomplished in 
what order. In short, planning is a method for ensuring that the 
activities will be conducted in the most efficient manner. 

This chapter describes major factors that should be con¬ 
sidered in scheduling a research program. Sections of this 
chapter, particularly those dealing with methodology, are 
expanded in later chapters. Obviously, the nature of the 
research will dictate the specific steps that must be undertaken 
in each type of research project, and no single plan of attack or 
schedule will satisfy every requirement. However, requirements 
of all research projects can be addressed at a general level. These 
general requirements are the framework for identifying addi¬ 
tional requirements unique to any specific research project. 

Factors to Consider in Planning 
Data Collection 

Factors to consider in planning the data collection phase of 
the research project relate to sources of data, to the methods 
required for obtaining or extracting the data, and to the 
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logistics and coordination required to gain access to the data. In 
planning the data collection phase, the researcher should ask 
these questions: 

Sources 

Do the data exist, or must they be “generated”, i.e., 
collected from primary sources or by firsthand observation? 

Who or what are the sources of data? For example, will 
primary or secondary sources be used? Are the sources 
human, documentary, or in some other form of record? 

Will any transformation of the data be required (e.g. from 
magnetic tape to a printed page)? 

Where are the data located (e.g., at U.S. Government 
facilities, private or state-supported institutions, in data 
“banks”, libraries, archives, etc.)? 

How can the data be used or obtained? Must they be used at 
another facility? Can they be borrowed? Do they require 
special facilities for utilization such as microfiche readers or 
other display devices? 

What are the priorities for locating and obtaining data? 

Obtaining data 

Will the data be collected from whole populations or from 
representative samples of larger populations? Will a poll or 
survey be required? 

If samples of larger populations are to be tapped, what 
method will be used for collecting data: personal interviews, 
mail questionnaires? 

How will the sample be constructed? 

What size will the sample be? 

What measurement devices will be used (e.g., questionnaires, 
rating scales, etc.)? 

Are the measurement devices “off the shelf’ items, or must 
they be constructed? 
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II' the devices are to be constructed, have provisions been 
made to validate them? 

I lave the measurement devices (survey instruments, record¬ 
ing formats, etc.) been pilot tested? 

What types of forms should be used for recording data 
(other than interview schedules and questionnaires)? 

Can the data be pre-formatted for ease in later analysis? 

Will any special equipment be required to collect the data 
(e.g., tape recorders, cameras, etc.)? 

If other people will assist in data collection, will training be 
required (e.g., dry runs, practice sessions)? 

Logistics and coordination 

How will first contacts be made? In person, by telephone, 
by letter? 

Will approval from higher authorities be required before 
visiting sources or repositories of data? 

By what means is the approval obtained? 

Are there any protocol considerations to take into account? 

Will travel be required? If so, by what means? How much 
funding will be required? By whom will the funds be 
provided? 

With respect to travel, are there any special requirements 
such as passports, visas, or immunization? 

Will clearances be required or must need to know be 
established? 

Who are the contact points of the agency to be visited? 

Must any documents such as form letters, questionnaires, or 
other data collection formats be printed or reproduced for 
the data collection effort? If so, by what means, where, and 
by whom will they be prepared? 
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Factors to Consider in Planning 

Data Analysis 


In nearly every type of research, data analysis can (and 
should) begin as soon as the first piece of information is 
collected. Therefore, in the planning stages, attention should be 
paid to the manner in which the data are to be collated and 
processed once they are obtained. Regardless of the nature of 
the research, the analysis phase always requires collation . 
Collation may involve comparing the contents of various 
documents, relating the contents of documents to maps or 
charts, or collation may simply involve ordering data chrono¬ 
logically, or topically, or by subject or author. Sometimes 
collation is performed only on the “externals” of the data; for 
example, on the author, date, subject, and place of publica¬ 
tion. 1 Again, the nature of the collation (and the analysis) to be 
performed will be determined by the nature of the research 
problem. Various kinds of analytic operations that can be 
performed on data will be discussed in more detail in later 
chapters, but at the planning stage, attention should be paid to 
the following factors: 

Has a data storage and retrieval system been established? 
(Systems can range from a 3 x 5 card file to a computer’s 
data base.) 

Are there secure places for storing data (if the data are 
classified)? 

Will any data reduction be required? For example, will there 
be large amounts of statistical data to be taken off 
questionnaires or rating scales and tabulated by categories? 


^‘Collation” requires better definition. For some analysts, collation 
relates only to the process of bringing together the information or 
organizing and cataloging the data. To other analysts and researchers, the 
term collation is tantamount to the whole analytic process. In the sense 
that the term is used in this chapter, collation refers to the process of 
ordering, indexing, and organizing information prior to further analysis 
and synthesis. 
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Will data processing equipment be required? If so, what 
kinds will be required, where will they be available, and how 
much computer or machine time will be required. 

How much lead time will be required to schedule equip¬ 
ment? 

What special research aids will be required; e.g., maps, 
charts, photographs, atlases, gazetteers, and so on. 

Will basic reference documents such as statistical handbooks 
and biographic registers be available when they are needed. 

Is there sufficient lead time to order them? 

Do algorithms or models exist, or must they be developed 
before the data can be used? 

Are hypotheses formulated to a degree of specificity so that 
analysis can proceed? 

Will personnel with specialized skills be required (e.g., key 
punch operators, programmers, photo interpreters, or con¬ 
sultants)? How much advance notification will these people 

require? 

Will all of the personnel required to analyze the data have 
appropriate clearances? 

Factors to Consider in Planning 

Report Preparation 

Because report preparation is the last phase to be com¬ 
pleted, it is typically the phase that is shortchanged most often 
in terms of insufficient time, funds, manpower, or resources. 
This is unfortunate. The report preparation phase is the 
culmination of the entire project; furthermore, the product of 
this phase represents the only tangible evidence of the effort 
expended over weeks, months, and perhaps years. Unfortu¬ 
nately, by the time this phase is reached, the researcher has very 
little flexibility for re-programming. He is committed o 
decisions made during the earlier phases. It is for this reason 
that the researcher must pay particular attention to the 
following factors: 
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What format should be used? (Assigned projects usually 
require adherence to an established format; self-initiated 
projects often permit the researcher to establish his format. 
Institutional requirements must be considered.) 

How long will the report be? 

For what purpose will the product be used? For general 
information, for policy planning, for decision making in a 
very specific context? 

Who will the readers be? Will the report be read primarily 
by highly trained specialists in the field or by “generalists”? 

Will the outline prepared during the problem definition step 
suffice? Must it be modified or rewritten completely? 

How detailed should the report be? 

Will “art work” be required; e.g., statistical tables, illustra¬ 
tions? 

How much lead time will be required for preparing the art 
work? Who will do it? Where will it be done? How much 
will it cost? How long will it take? 

Will photographs be included? If so, where or how will they 
be obtained? 

How much time will be required to produce the first draft? 

Who will edit the report? How long will the editorial review 
take? Will there be time for revision? 

How much time will be required for final printing? 

Are there any special format or stylistic requirements 
pertaining to footnoting, use of special terminology, biblio¬ 
graphies, or layout? 

Are there basic reference books on style and usage 
available? 
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Wo rk Breakdown Structure: The 
I i rst Cut at Scheduling 

Regardless of the complexity of the project, it would be 
well for the researcher to prepare a work breakdown 
,s an aid in identifying what tasks must be accomplished in 
what sequence. Preparing a work breakdown structure involve 
breaking down the overall project into a senes of smaller, more 
manageable tasks. The subdivision of a project mto tasks 
continues until the desired level of detail is cached In highly 
complex projects that involve many steps and q 
coordination of other workers or equipment, the work break 
down structure may have three or four levels of complexity. 
I or a one-man project that is fairly uncomplicated a two-level 
breakdown may suffice. A work breakdown structure for the 
data collection phase of a project might look like this. 

DATA COLLECTION PHASE 


First Level: 

REViEW LITERATURE 

(TaSk Z) (Task 3) 


Second Level: 


2.! REVIEW UNCLASSIFIED 3.1 MNTOVKEY 
PERIODICALS PERSONNEL 

2 2 REVIEW CLASSIFIED 3.2 COORDINATE VISITS 
DOCUMENTS 


2.3 REVIEW UNCLASSIFIED 
BASIC WORKS 


3.3 CONDUCT INTER¬ 
VIEWS 


For more precise planning, the researcher could break down 
each task into further steps, for example: 

2 1 REVIEW UNCLASSIFIED PERIODICALS 
FROM-TO- 


2.1.1 CHECK INDEXES/ ABSTRACTS 

2.1.2 SURVEY DATA BASES 
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2.2 REVIEW CLASSIFIED HOLDINGS 

2.2.1 CLASSIFIED PERIODICALS 

FROM-—TO- . 

2.2.2 CLASSIFIED BASIC SOURCES 
(e.g., NIS’s . . latest date) 

2.3 REVIEW UNCLASSIFIED “BASIC WORKS” 

2.3.1 INHOUSE DOCUMENTS 

2.3.2 LIBRARY OF CONGRESS 

2.3.3 STATE DEPARTMENT LIBRARY 

3.1 IDENTIFY KEY PERSONNEL 

3.1.1 U.S. DOD PERSONNEL (e.g., defense 
attaches) 

3.1.2 U.S. STATE DEPARTMENT PERSONNEL 

3.1.3 DIA ANALYSTS 

3.1.4 CIA ANALYSTS 

3.2 COORDINATE VISITS 

3.2.1 LOCATE KEY PERSONNEL 

3.2.2 CLEAR VISITS WITH APPROPRIATE 
AUTHORITIES (for foreign nationals,e.g.) 

3.2.3 MAKE INITIAL CONTACTS WITH KEY 
PERSONNEL 

3.2.4 SEND CLEARANCES 

3.2.5 PREPARE INTERVIEW SCHEDULE OR 
QUESTIONNAIRES 

3.2.6 REQUEST TRAVEL AUTHORIZATION 
AND FUNDS 

3.3 CONDUCT INTERVIEWS 

3.3.1 WRITE UP RESULTS OF INTERVIEWS 

3.3.2 FOLLOW-UP INTERVIEWS IF NECES¬ 
SARY 
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Plotting Activities on a Time Line: 

T he Second Cut at Scheduling 

Having identified the" tasks to be performed, having 
identified the tasks upon which subsequent tasks are based, and 
having estimated the time required to complete each task 
researcher should plot the activities along a time line. The time 
line shows the amount of time available to complete the 
project. At this step it is helpful to work backward from the 
'late by which the project must be completed. The first plot 
might look like this one shown in Figure XI-1. 

Obviously, in the example just shown the researcher 
would not have enough time to complete the project in the 
manner he depicted. In instances such as this the 
might examine his topic and further delineate the topic, or 
request additional time or manpower. But the plan as show 
not very realistic. For instance, the plan shows that , 
subsequent task will be started until a prior one had been 
completed. This is not efficient planning, and it leaves very little 

room for error. 

A more realistic and efficient plan is shown in Figure 
XI-2 In this plan, data collection begins before the problem 
definition phase ends. This is not unrealistic. Although some 
refinement^ may be made in the definition of the problem as 
he project progresses, it is unlikely that the entrre problem 
would change. Therefore it would be to the researcher s 
advantage to begin data collection as soon as possible. Furth 
more there is no reason why analysis cannot begin as soon as 
data become available. Finally, report preparation can begin 
while the data are still being analyzed. Every research report 
contains a certain amount of “boi er plate nratenal-mtro- 
ductory material relating to the statement of the problem, 
assumptions and hypotheses, methods used, and often reviews 
of literature. These portions of the report can be prepared 
before all of the data are collected and analyzed. 

Bv overlapping the tasks, the researcher can (.ptii.iiac liis 
time and manpower utilization. Furthermore, the sooner tasks 
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Figure XI-2. Phases of a Project Plotted on a Time Line, 
(Revised.) 
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Figure XI-3. Second and Third-Level Activities Plotted 
on a Time Line. 


arc started, the sooner problems become apparent. And the 
sooner a problem is identified, the greater are the chances of 
solving it within the time available. 

Having blocked out the major phases of the project, the 
researcher is ready to examine more closely each specific phase. 
Again, for greater specificity, the researcher might want to 
schedule second or third level events. Considering only the data 
collection phase, his schedule might look like this (Figure XI-3): 

Use of Networks in Planning 

Complex Programs 

Research projects that involve several researchers and 
many operations, some of which are dependent upon others, 
may be sufficiently complex to warrant more detailed planning. 
In these cases, it may be helpful to prepare a network of 
activities and events for each major task of each phase, and then 
prepare a master network showing the relationship of each task 
to the overall effort. 

One way of preparing a network is to make a flow 
diagram of the activities to be performed. A simple flow 
diagram of the data collection phase of a project is shown in 
Figure XI-4. 

Another way of constructing a network is to depict 
graphically all of the tasks on the work breakdown structure by 
using a “bubble” to represent the start or the termination of an 
activity and a line with an arrow to represent the activity itself. 
Figure XI-5 is a network of the interview portion of the data 
collection phase discussed previously. 2 


2 In this example, the “bubble” indicates the start of an activity. The 
start of an activity is an “event.” Whether “bubbles” indicate the initiation 
or the termination of an activity is up to the planner, but the use must be 
consistent. 
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Figure XI-4. A Flow Diagram of Second-Level Activities 
in the Data Collection Phase 



This format is especially useful for showing dependencies, 
i.c., identifying those activities which must precede others; for 
identifying activities that can be performed simultaneously and 
for identifying critical events which, if not accomplished, could 
jeopardize the successful completion of the project. This 
network plot is not only a planning device, but it is also a useful 
device for measuring progress. Scheduling events in this manner 
is called PER T (for Program Evaluation and Review Technique), 
and in its most complex form involves making three time 
estimates for completing each step: an optimistic time (t Q ), a 
most likely time (t m ), and a pessimistic time (tp). Manually, or 
by use of a computer (for extremely complex programs), three 
time-estimates-to-completion can be made. 

But even without calculating time requirements, the 
network depiction is useful for showing where time could be 
saved or where activities could be performed concurrently. For 
instance, in the example shown in Figure XI-5, time could be 
saved (or perhaps delays could be avoided) by starting event 
3.2.6 (obtaining travel authorization and funds) as soon as 
initial contacts with key personnel had been made (3.2.3). In 
fact, preparation of an interview schedule or questionnaire 
(event 3.2.5) might start as soon as key personnel had been 
identified (event 3.1) as shown in Figure XI-6. 

The concept of PERT is simple: its application, however, 
can be very complicated. Researchers who want more informa¬ 
tion about PERT should check the numerous texts on the 
subject, such as Robert W. Miller’s Schedule, Cost, and Profit 
Control with PERT .3 Furthermore, students at the Defense 
Intelligence School have access to General Electric terminals 
and computer programs for PERT/CPM (Program Evaluation 
and Review Technique/Critical Path Method) should their 
project planning be sufficiently complex to warrant it. 


•^Robert W. Miller, Schedule, Cost, and Profit Control with PERT: A 
Comprehensive Guide for Program Management (New York: McGraw-Hill 
Book Company, Inc., 1963). 
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Figure XI-5. A PERT Diagram Showing all Tasks Relating to the Interview Portion of a Data Collection Phase. 

(Normally the “bubbles” would be numbered consecutively and would not contain captions.) 
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Maintain a Journal 


For the researcher with a fallible memory, a journal is a 
necessity. Daily journal entries help the researcher keep track of 
significant developments, appointments, names, telephone 
numbers, addresses, potential problem areas, approvals, and all 
ot the numerous logistic and administrative details that even the 
simplest research program requires. Files should be maintained 
tor all written correspondence, and logs should be kept of all 
research-related telephone conversations. The telephone log can 

be kept within the project journal if the project is a one-man 
effort. 


In addition to helping the researcher keep track of 
numerous details relating to his effort, the project journal can 
serve as a basis for planning other research efforts, for 
trouble-shooting (when problems arise), and for assuring super¬ 
visors that all necessary contacts were made. The journal should 
be kept unclassified. Classified or sensitive information should 
be kept in a separate file. 

A Word of Caution 

Planning is a means to an end: it is not an end in itself. 
Perfectionists who attempt to account for every contingency 
never get a plan off the ground. Planning should never be more 
complex than the project being examined. PERT, for example, 
is useful when unique, one-shot projects of a complex nature 
are planned.^ In order for PERT to be worth the effort, at least 
twenty events should be planned involving the activities of a 
minimum of two people, over a period of at least two months. 
Networks are useful in planning. But for simple, straightforward 
projects, a flow diagram might serve as well as the more 
complex PERT network. 


4 Such as building the first Polaris-type submarine, the project for 
which PERT was developed originally. 
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Summary 


t> All phases of the research project should be considered 
during the initial project planning activity. The phases, other 
than the problem definition phase, include the data collection 
phase, the data analysis phase, and the report preparation phase. 

> Factors to consider in planning the data collection phase 
include sources of data, methods to be used in obtaining the 
data, and logistics and coordination required to obtain the data. 

> Factors to consider in planning the analysis phase include 
the methods to be used in collating the data, as well as the 
methodologies to be used in the analysis of the data. Mechanical 
considerations relating to extracting and processing data, as well 
as theoretical considerations relating to model selection or 
development must be taken into account in this planning. 

> Factors to consider in planning the report preparation 
phase relate to determining the purpose for which the report 
will be used, the readers, approximate length of the report, as 
well as standards relating to style, organization, and form and 
formats of the report. 

> Planning involves scheduling. The first scheduling activity 
for a large-scale research effort is preparing a work breakdown 
structure and then plotting the activities on a time line. If a 
research program is complex, it may also be necessary to make a 
network or flow diagram of activities. The complete schedule 
should indicate the activities to be performed, the sequence of 
activities, and dependencies between and among activities. 

t> One-man research programs require relatively simple 
schedules. But large-scale research programs require detailed 
schedules and plans and may necessitate the use of program 
planning techniques such as PERT or PERT/CPM. 

t> Maintaining a journal helps the researcher keep track of 
the myriad details involved in planning and conducting the 
day-to-day research activities. 
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CHAPTER XII. 

COLLECTING THE DATA-PART ONE: 
EXPLOITING THE LIBRARY 


A man will turn over half a library to make one 
book. 

Samuel Johnson 

On the night of March 20, 1935, Berthold Jacob, a German 
journalist living in London, was kidnapped from a restaurant in 
Basel, Switzerland, and taken to Gestapo headquarters in Berlin. 
Berthold’s offense: publishing a book about the German 
General Staff, a book so detailed that even rifle platoons 
constituting the newly formed Panzer divisions were identified. 

How did he get his information? Who were his sources? 
Where was the security “leak”? These were questions the 
interrogators asked Jacob. The answer, to the consternation 
(and grudging admiration) of the interrogators, was the 
“German press.” Over a period of months and years, Jacob 
noted, recorded, and compiled press releases, obituaries, 
wedding announcements, and articles from German military 
journals. No leaks existed, no agents, and no informers were 
involved-only a body of “open” source material available to 
anyone with an “analytic mind.” 1 

The intelligence researcher has access to vast amounts of 
open source and classified materials. This chapter describes how 
to use the type of repository in which these materials are 
stored, the library. 

Most research projects start in a library and end in a library. 
Even research of an experimental nature, or research that 
involves polls or surveys, requires some preliminary library 


1 Ladislas Farago, War of Wits: The Anatomy of Espionage and 
Intelligence (New York: Funk & Wagnalls Co., 1954), pp. 55-58. 
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research. Before a manuscript is submitted for final publication, 
footnotes and citations must be checked in the library, and, 
assuming that the research product was acceptable, the library 
will be the final repository of the document. 

Every library has unique features. Ground rules for using 
one library might not apply to another. But in general, libraries 
have the same essential elements: a system of procurement, a 
system of classification (which includes storage and retrieval); a 
system of circulation (or dissemination); and a system tor 
providing reference assistance to the user. 

The researcher should be familiar with all of these 
functions. For example he should be familiar with the guide¬ 
lines used by the library’s staff for selecting materials and the 
procedures for obtaining materials from other libraries; he 
should know how the materials are cataloged so that he can find 
what he needs; he should know which materials circulate and 
which materials have to be used in the library; and he should be 
familiar with the reference works and services that the reference 
department can provide. 2 

Library research involves three stages: an exploratory stage 
in which materials are identified and located; an investigative 
stage in which located materials are examined; and a recording 
stage in which information and data are extracted and recorded 
for"later use. 3 The stages occur in this sequence, and the stages 
may have to be repeated in each library visited. The stages are 
described below. It should be noted that much of the first 
stage-the exploratory stage-may have been performed during 
the initial planning phase of the project, and that the data 
collection phase of the project could very likely start with the 
investigative stage of the holdings in a library. 


2 ln small libraries, the reference department might constitute the 
entire staff. 

3 Henry Lester Smith and Johnnie Rutland Smith, An Introduction to 
Research in Education (Bloomington, Indiana: Educational Publications, 
1959), p. 75. 
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The Exploratory Stage 

Card catalog 

The objective of the exploratory stage is to identify ant 
locate materials relating to a specific topic. In order to optimize 
the search, the card catalog should be used. The card catalog is a 
listing of every volume contained in a library. Separate catalog 
cards are prepared for the author, for the title of a work, and 
for the subject. The library of the Defense Intelligence School 
for example, has all subject cards arranged alphabetically in one 
file, and title cards and author cards interfiled alphabetically in 
another file. 


All cards contain a call number in the upper left-hand 
comer by which a book can be identified. Different libraries 
may use different systems of classification. The School’s library 
(and most U.S. Government libraries) use the Library of 
Congress Classification. This classification system identifies 
subject area groupings by letter, for example: 


A — General works 
D — History except American 
DK Russia 
DS — Asiatic countries 
DT — Africa 
DU — Oceanic countries 
E — General American and 
U.S. history 

F Canadian and Latin 
American history 
G Geography 


H — Social Sciences 
HB-HF — Economics 
HX — Communism 
J - Political Science 
K — Law 
Q Science 
T — Technology 
U — Military science 

UB Military Intelligence 
V — Naval science 
Z Bibliography 


The Dewey Decimal System, the other major classification 
system that may still be used by some libraries, uses numerals to 
identify major groups of subject matter; for example: 


000 General Works 
100 Philosophy 
200 Religion 
300 Sociology 
400 Philology 


144 


500 Natural Science 
600 Useful Arts 
700 Fine Arts 
800 Literature 

900 History, Travel, and Biography 

Few researchers, other than librarians, memorize the 
classifications. But the researcher should memorize at least 
I hose letters or numbers pertaining to his area of specialization. 

I ibrary of Congress and Dewey Decimal System identifiers are 
identical in all libraries; therefore, a little effort spent in one 
library increases one’s proficiency in another. 

Catalog cards are used primarily for identifying works and 
lor helping the researcher (or the librarian, if the stacks are 
“closed”) locate a specific work. But the cards also contain 
much information that is valuable to the researcher. For 
example, the researcher might determine from the subtitle of a 
hook that the contents probably would not be appropriate for 
his needs and, therefore, would not warrant much attention 
during his first examination of materials. Interpreting the cards 
can save the researcher hours of effort in obtaining works that 
were outdated, probably unreliable, or inappropriate. 

A word of caution: because a library’s catalog does not 
contain an entry for a specific topic does not mean that no 
books on that topic exist. To determine what books are still in 
print, or books that had been published in the past, the 
researcher should consult the Subject Guide for Books in 
Brin t— the current edition as well as earlier editions. Further¬ 
more, bibliographies in books that are available may suggest 
other works that might be useful. 

Indexes to periodicals 

The card catalog is the best source of information relating 
to a library’s holdings of bound works. For unbound 


4 One of the more informative and entertaining descriptions of how 
the researcher might “exploit” the card catalog is given in Barzun and 
Graffs The Modem Researcher, pp. 69-75. 
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periodicals, however, and for information relating to curreni 
affairs, or to scientific and technological developments, the can I 
catalog will be of little use. In these instances the researcht 
should use various indexes to periodicals. The two mosi 
commonly used indexes are the Reader's Guide to Periodical 
Literature and The New York Times Index. 

Not to be overlooked are indexes of periodicals relating tc 
various disciplines—the Engineering Index or the Psychological 
Index , for example, or the Air University Index to Military 
Publications . Most professional journals also publish annual 
indexes. 

General reference works 

Unless a researcher is thoroughly familiar with his topic, he 
will usually start his investigation by examining general re¬ 
ference works. The most commonly used general reference 
(aside from a dictionary) is the encyclopedia. Encyclopedia 
entries are prepared by scholars in the field, are generally 
updated with each new edition, and often contain a short 
bibliography of basic works relating to the topic. For these 
reasons, encyclopedias are most useful at the outset of a 
research project. 

However, there is much variation in encyclopedias, not only 
among different encyclopedias, but also between different 
editions of the same work. For example, the eleventh edition of 
the Encyclopaedia Britannica is an exceptionally good source 
for historical information, and these editions are collectors’ 
items. On the other hand, one popular encyclopedia (and its 
companion children’s version) contains numerous errors, for 
example: describing the Orinoco River as emptying into the 
Caribbean, and depicting Lord Nelson at Trafalgar, clutching his 
chest with his right arm while propping himself on the deck 
with his left.^ 

Inexpensive but useful general references are world 
almanacs published by The New York Times or by the 


^Nelson lost his right arm following the battle of Santa Cruz in 1797. 
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Newspaper Enterprise Association, Inc. For quick reference on 
I. ,m, information, even old editions are useful. 

'.pocific reference works 

Specific reference books include numerous dictionaries 
,, technical dictionaries), various versions of Janes co 
pi iidia (Janes' Fighting Ships, Janes’ All the 

Hid Janes’ Weapons Systems); y e f ooks .l e ' g :£ n S ' “ ( " 

I .rhnok the Europa Yearbook; biographical dictionaries (e.g., 
;,;;tmy versions of Wo'. Who); and atlases and gazetteer, 

I urlhermore, every profession has its basic works, and 
Intelligence researcher who specializes in a given area soo 
learns which references are especially useful. 

Classified libraries 

Undoubtedly, the intelligence researcher will need to 
examine classified information in the course of his research. 

( lassified holdings have unique classification systems because 
ihe classification system number is sometimes used as a 
document control number as well. The classified collection o 

MIC Defense Intelligence School library, for ex ™P le ' “X" t 
|,V document (DIL) number, and the special collection 
„ranged in a manner that supports the course requirements of 
Mm school Every intelligence organization has special groun 
esTor determining who has access to classified in ormahom 
ami appropriate clearances are a necessary (but not always 
sufficient) qualification for gaining access to the data. 

Open stacks and browsing 

Inefficient as the process may be, there is something to be 
said for browsing through open stacks when 8™un ™ ® s P® r ( 
it Sometimes titles of books or documents not directly tor 
seemingly not) related to the research topic act as stl "J u l °^ 
recalling possible solutions to problems or suggest addition^ 
information that would no, he found in —onal 

sources. Sometimes information is lost in the process ot 
reducing the contents of a book to a numeral or letter for 
classification purposes, and without direct access to a work, 
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valuable information might never be discovered. Browsing is 
notoriously unsystematic and time consuming and perhaps i 
better used to polish a report than to produce the first draft. 

Automated systems 

Where access must be gained to large data bases whose 
volume increases daily, automated systems provide the only 
feasible method. For example, students of the Defense Intelli¬ 
gence School have access to two million intelligence informa¬ 
tion reports (IR’s), some of which go back to 1900. Access to 
these documents is available through the IRISA (Intelligence 
Report Index Summary File) and the IRFLA (Intelligence 
Report File Archive) at the main DIA library at Arlington Hall. 
The automated systems are methods not only for locating 
documents, but also, in many cases, for examining their 
contents. 

The Defense Intelligence School has four automated re¬ 
trieval systems for student use: the DIAOLS/COINS system, the 
CIRCOL/CIRC II system. The New York Times Information 
Bank, and a General Electric time-sharing system. 

DIAOLS (DIA On-Line System) is a time-sharing system 
that provides both remote and local batch-processing services on 
DIA files. COINS (Community On-Line Intelligence System) is 
an intelligence community-wide network of computer-based 
information retrieval systems. DIAOLS terminals provide access 
to the files of COINS. 

CIRCOL system (Central Information Reference and Con¬ 
trol On-Line) and the CIRC II system provide on-line support 
for analysis or research purposes. The data bases of these 
systems contain abstracts of open source documents as well as 
finished and unevaluated intelligence documents pertaining to 
scientific and technological developments in Soviet-bloc 
countries. System outputs are abstracts in the form of hard¬ 
copy printouts. Complete texts of documents are contained on 
microfiche. This system is maintained by the Foreign Techno¬ 
logy Division at Wright Patterson Air Force Base. 
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flic New York Times Information Bank is a system for 
i* moving abstracts of news and editorial matter published in 
II,, New York Times and in some sixty other periodicals. 
\hslracts of requested subject matter are projected on a CK 
display and a hard copy can be produced by a high-speed 
pi inter. Complete texts of the referenced and abstracted 
newspaper articles are contained on microfiche files. 

The General Electric system has both time-sharing and 
I >aI ch-processing capabilities. The system can be used for 
maintaining data files, and for internal management purposes, 
lot example: PERT/CPM, statistical analysis, and resource 
allocation. As such, it is more than a retrieval system. 

Where to go for more information 

Since even small libraries are often parts of larger libraries, 
nul since the holdings of even small branch libraries are 
potentially unlimited when inter-library loan agreements exist, 
,l is easy for the researcher to be overwhelmed by the 
magnitude of the resources available to him. For this reason it 
may be helpful for the researcher to peruse Louis Shores Basic 
Reference Sources 6 or Contance M. Winchell s Guide Jo 
Reference Books 1 before he begins his data collection. Essential 
reading for students of the Defense Intelligence School is 
“Library Resources and Services in Support of the Master s 
Degree Program in Strategic Intelligence. This document 
describes the organization, function, and facilities of the School 
library as well as the main DIA library at Arlington Hall to 
which the students have access. 


^Louis Shores, Basic Reference Sources (Chicago: American Library 
Association, 1954.) 

^Constance M. Winchell, Guide to Reference Books (Chicago: 
American Library Association, 1967). 

8Defense Intelligence School, “Library Resources and Services in 
Support of the Master’s Degree Program in Strategic Intelligence, n.d. 
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The Investigative Stage 

The investigative stage of library research involves tw( 
operations: a preliminary examination of materials in ordei 
to determine which sources are most pertinent, complete, and 
authoritative; and a critical examination of selected works. 

Preliminary examination 

Speed is essential in the preliminary examination. The 
researcher rarely has time to read everything that may be 
available on a subject. Time is saved in the preliminary 
investigation by examining the “skeleton” of a text: first, the 
“externals”—author, co-author, date and place of publication 
then the “internals”—the preface, foreword, and especially the 
table of contents, section headings, and index. 

Inexperienced researchers typically ignore prefaces and 
forewords in their haste to get on with the work. This is a 
mistake. Often the preface - that introductory portion of a book 
prepared by the author— tells what the writer was attempting to 
achieve by writing the text, what was omitted, why a certain 
approach was taken, and states the assumptions underlying the 
work. Forewords-introductory material usually prepared by 
someone other than the writer—often place a work in a setting 
or perspective. When forewords are written by authorities in the 
field, the forewords may contain important information that is 
addressed in the text. But admittedly, many times forewords 
are nothing more than glowing tributes to an author. The ability 
to “skim” a text is invaluable to a researcher in this preliminary 
examination of materials. 

Critical examination 

Works that survive the first screening are subjected to a 
more critical examination—the examination that really con¬ 
stitutes the major portion of the activity known as data 
collection in archives. Critical examination requires that the 
reader constantly asks himself the same questions he would ask 
if he were examining an intelligence report (which, indeed, he 
may be doing), namely: how reliable is the source and how well 
does the information correspond to other information or data 
contained in other sources? 
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I he Data Extraction Stage 


Books and other reference materials are examined in order 
to l ind information relevant to a topic or a problem area. en 
needed information is located, it must be extracted for use in 
later analysis. In library research, data extraction is synonomous 
with notetaking. Notes are taken in order to record substantive 
information for later inclusion in the text of the report and for 
proper source attribution. 

For notetaking, any number of possible combinations of 
cards, notebooks, looseleaf binders, and so on, may be used. 
But most researchers find it useful to keep two separate tiles: 
one file containing nothing but the bibliographic information 
on sources, and the other file containing subject matter content. 
The second file is cross referenced (by author, or by some 
arbitrarily assigned number) to a source. Having two separate 
tiles permits the researcher to check on, say, a publication date 
of a reference without having to plow through sheaves of notes 
on subject matter. Furthermore, some writers organize topics or 
information by physically arranging their notes. In this case, 
cards containing bibliographic information would add con¬ 
fusion. In order to use cards for organizing material, however, 
only one topic may be recorded on each separate card. This is 
an inviolable rule. 

The manner in which notes are recorded is a personal 
matter. Some researchers use 3 x 5 cards; others use 5 x 6 or 
5x8 cards. The larger sizes are better for recording long 
quotations, but they tend to be bulky for recording on y 
bibliographic data. For this reason, some writers use the 3 x 5 
cards for bibliographic data and the larger cards for notes. 
Again some researchers use cryptic characters, shorthand, ey 
words, and so on, whereas other researchers laboriously copy 
verbatim from the text, regardless of how the material will be 
used ultim ately in the report. 9 

9when the author’s exact words are to be quoted in the research 
report and when bibliographic data are extracted from the book, it is 
imperative that the text be copied verbatim. For other purposes, 
paraphrasing the text normally suffices. 
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Information about a source must be accurate, and it must 
be complete. Errors in source data reflect poorly on the 
researcher’s competency and credibility, and they can be costly 
and inconvenient, especially when the researcher no longer has 
the original source document for reference. Experienced re 
searchers become compulsive about recording complete biblio¬ 
graphic data, often because of some unfortunate experience in 
the past. 

One time-saving procedure in extracting bibliographic data 
is to record the information exactly as it would appear in the 
bibliography of the report. Then, in final report preparation, 
the bibliography can be typed directly from the cards. 

The medium by which notes are recorded is another 
personal matter. Pencil marks rub off and render cards 
unintelligible, but some archives do not permit pens to be used 
because of the danger of indelibly marking original source 
documents. Lightweight, portable typewirters are good for 
taking legible notes quickly, but even the smallest machine is 
bulky, cumbersome, and noisy, and often may not be used in 
reading rooms. 

Regardless of the systems or sets of procedures used, they 
must satisfy the needs of the researcher. Notetaking is a serious 
business, and it must be done systematically and consistently. It 
is during the notetaking process that errors are often introduced 
which may be embarrassing at later stages. 

Three sources for additional information on notetaking are 
Earle W. Dow’s Principles of a Note-System for Historical 
Studies ,10 Catherine Drinker Bowen’s Adventures of a Bio¬ 
grapher, 1 1 and Barzun and Graffs The Modern Researcher. 1 2 

^Earle W. Dow, Principles of a Note-System for Historical Studies 
(New York: The Century Co., 1924). 

^Catherine Drinker Bowen, Adventures of a Biographer (Boston: 
Little, Brown and Company, 1959). 


1 ^Barzun and Graff, The Modern Researcher. 
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Summary 

t> |'he library is the basic repository of information used by 
vnlually every researcher in at least one phase of his research 
project. 

> Library research involves three stages: an exploratory stage 
m which data sources are located and identified; an investiga¬ 
tive stage in which the sources are examined-first, cursorily, 
then critically; and a data extraction phase in which data are 
copied or paraphrased from sources for further analysis or 
inclusion in new text. Each stage requires certain skills. 

t>ln addition to catalogs and standard references found in 
most libraries, students of the Defence Intelligence School have 
access to automated retrieval systems which permit the students 
to draw on the data resources of various members ot the 
intelligence community. 

> Although each library has specific ground rules, most of the 
techniques and procedures for locating and obtaining data are 
similar for all libraries. Skills in using card catalogs, sta n da ™ 
reference works, and indexes are transferable skills and should 
be developed by all researchers. 
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CHAPTER XIII. 

COLLECTING THE DATA-PART TWO: 
FIELD DATA COLLECTION 
TECHNIQUES 


In the earlier years we had read too much faulty military 
history: the combat portions rarely rang true; they were 
overromanticized, inconsistent with human nature , or 
lacking in decisive detail. Also , in the crisis of action, the 
field frequently became obscured, and the historian 
made the sad confession: “What then happened was 
hidden by the fog of war. ” We began with the simple 
conviction that there must be a way to dissipate that 
fog. 

S. L. A. Marshall 

As valuable as documents and records are to the intelligence 
researcher, there are instances in which already recorded data 
may not suffice. For example, a defector may not have revealed 
important information simply because no one requested it; or a 
recent visitor to a “denied” area may have new information not 
yet contained in any printed document; or two accounts of an 
operation may differ significantly on an important detail. When 
it becomes necessary to dissipate “the fog of war”, the 
researcher may have no other recourse but to collect informa¬ 
tion directly from the source. The procedures used to collect 
information directly from the sources will be referred to as field 
data collection techniques. 

This chapter discusses those field data collection techniques 
commonly used by social scientists and which, with appropriate 
modification, can be used by intelligence researchers as well. 
Knowledge about field data collection techniques is important 
for two reasons; first, the researcher may have to employ one or 
a combination of the techniques in his own project; second, the 
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researcher must know the strengths and weaknesses of the data 
collection techniques in order to assess the quality of the 
information they yield. 

Field data collection techniques that will be discussed in 
this chapter include the interview (and its variations); observa- 
liorial methods; questionnaires; mass media appeals; and the use 
() f panels. Another technique of inquiry discussed in this 
chapter cuts across other techniques and may involve more than 
direct contact with original sources. This technique is a 
variation of the critical incident technique. 

The Interview 

Interviews are methods of eliciting information directly 
from a source generally in the context of a face-to-face meeting. 
Done well, an interview is more than a guided or unguided 
two-way conversation. At its best, an interview involves noting 
and interpreting inflections, expressions, pauses, covert 
glances—the whole gamut of human responses-as well as the 
oral responses to questions. 

In intelligence jargon, professional interviewers are called 
interrogators. Skillful interrogators are part clinician and part 
subject matter specialists who have honed the interview 
technique to a fine art. It is highly unlikely that the intelligence 
researcher will require the skills of the professional interrogator. 
But he should be aware of the varieties of interviewing 
techniques that exist because the interview is one of the more 
commonly used methods of eliciting information from human 
sources in intelligence. 

Interviews are used in large-scale operations, such as 
Operation Wringer, in the systematic interrogation of prisoners 
of war, as well as in small operations, for example: debriefing an 
attache, a traveler to a denied area, or a defector. In large-scale 
operations, interviews may be highly structured in the sense 
that specific questions formulated in advance of the interview 
are asked of all respondents. Highly structured interviews are 
appropriate when large amounts of specific information must be 
elicited from many respondents. Highly structured (or 
“directed”) interviews sometimes take the form of an orally 
administered questionnaire. 
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Unstructured (or “nondirected”) interviews-interviews in 
which the questions to be asked evolve naturally in the course 
of a discussion -are appropriate when general information 
relating to broad topic areas is to be elicited from one or several 
respondents. In either situation, the researcher must do his 
homework well in advance so that he recognizes important 
information when he hears it. 

The type of interview most likely to be used in intelligence 
research is a variation of the focused interview. Focused 
interviews are interviews conducted with persons known to have 
been involved in specific situations. Focused interviews require 
the use of an interview guide or “schedule,” and are directed 
toward specific topics about which something is already known 
by the researcher. 1 

A technique of interviewing that may also be useful for the 
intelligence researcher involves aided recall. In aided recall, the 
interviewer might show the subject a series of photographs or 
objects, or describe an event or situation, and then concentrate 
on the items that were recognized, recalled, or identified by the 
subject. For example, an attache who may have passed near a 
restricted area might be shown a series of photographs of 
objects or structures of the type that were believed to exist in 
that area and asked if he recalled seeing similar structures. 
Police lineups are forms of aided recall. 

At times it might be desirable or necessary to interview 
several subjects at one time. In these instances, comments by 
one respondent may stimulate recall by others. When issues very 
sensitive to the subjects are addressed, however, the presence of 
others may inhibit the respondents. Furthermore, when atti¬ 
tudes are addressed in group interviews, it is not uncommon to 
find that a “group” attitude evolves that does not accurately 
reflect the real feelings of the individuals that constitute the 
group. 


*In its clinical application, the focused interview addresses primarily 
the subject’s attitudinal and emotional responses to a specific situation. 
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All data collection involves a certain amount of data 
recording, and interviews are no exception. Information can be 
recorded in writing by the interviewer or by use of tape 
recorders. When the words of the respondent are to be quoted, 

1 1 is essential that the words are recorded verbatim, and for this 
reason tape recorders are especially useful. The use of a tape 
recorder also permits the interviewer to maintain eye contact 
with the subject and to concentrate on responses being made. 

I lowever, in instances in which comments “not for attribution 
are made, the presence of a tape recorder might inhibit a 
respondent. 2 In instances in which the respondent was under 
no obligation to reply, his permission should be obtained before 
recording his comments. 

Protocol considerations and methods of building and 
maintaining rapport with the respondent are beyond the 
purview of this text, but they are critical, nevertheless. In 
situations or in cultures where courtesy is especially important, 
an inadvertent slight on the part of the researcher might destroy 
rapport before the interview even begins. 

Although interviews typically involve face-to-face contact, 
there may be instances in which travel would be impossible, or 
when not enough time would be available to visit the subject 
personally. In these instances, telephones can sometimes be 
used. (Journalists do it very often when a statement is needed 
quickly from an agency’s spokesman.) Telephone interviews do 
not permit the interviewer to note important physical reactions 
of the respondent, and they often tend to be stilted. However, 


2 One of the authors used tape recorders extensively in recording 
subjects’ responses to questions dealing with drug use and abuse within 
their (Army) units. After a few moments, the subjects invariably ignored 
the recorders and made very candid comments. The subjects, it should be 
noted, had been granted immunity from prosecution, and their responses 
were recorded anonymously. 
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at times telephone interviews may be the only practical method 
to obtain needed information. 3 

The interview is a highly flexible and adaptable technique 
It permits the researcher to adapt his questions to the peculiar 
requirements of the individual respondent; it permits the 
interviewer to assess reactions to questions and to judge the 
credibility of the responses. Contradictions in testimony can be 
clarified on the spot, and the interviewer has flexibility to 
pursue unanticipated leads. 

The interview is not without its weaknesses, however. Every 
interviewer approaches a subject with an expectancy “set”, i.e., 
the interviewer has some expectation of what the respondent 
might say regarding a topic. This expectancy, plus the respon¬ 
dent’s tendency to avoid self-incrimination or embarrassment, 
may add up to what Pauline V. Young calls a “double dose of 
subjectivity.” 4 Interviews are expensive in time, and compared 
with other forms of eliciting information, the interview may be 
the most expensive form of data collection relative to the 
amount of information obtained. 

Interviews—even unstructured interviews—require planning, 
and mass interview efforts also require a considerable amount of 
interviewer training, dry runs, and rehearsals. Interviews are 
expensive. But at times they may be the only feasible method 
of obtaining needed information. 

Observational Methods 

Rarely do intelligence analysts and researchers have 
opportunities to make direct observations. But they often have 
opportunities for specifying how others should make and record 


3 In a sense, every telephone conversation an analyst or researcher has 
with his counterpart in another agency would qualify as a kind of 
interview. 

4 Pauline V. Young, Scientific Social Surveys and Research , (Engle¬ 
wood Cliffs, New Jersey: Prentice-Hall, Inc., 1966), p. 223. 
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observations. Observational methods are most appropriate for 
I'iilboring information about activities, events, objects, and 
t luiracteristics and behaviors of groups. 

Observational methods fall into two main classes: . uncon¬ 
trolled observations and controlled observations. As the name 
•a ingests, uncontrolled observations are observations made when 
,,„d where “targets of opportunity” present themselves. Uncon¬ 
trolled observations might be observations that a member ot a 
military advisory group or an attache might make as a spectator 

al a war game. , . 

One variation of the uncontrolled observation involves the 
observer as a participant in the activity, event, social process, or 
function being observed. An example of a participant observer, 
again, might be a member of a military advisory group who was 
assisting in training or in conducting operations with a foreign 
military force. A foreign service officer or a military attache 
attending a formal reception would be other examples ot 
participant observers. 

“Real time” imagery from remotely powered vehicles are, 
for the greater part, also uncontrolled observations. Uncon¬ 
trolled observations are made when unique phenomena are to 
be observed-phenomena whose characteristics, and in ac , 
whose very existence, may be unknown. 

Controlled observations, on the other hand, require the 
careful definition of the phenomenon to be observed, the 
standardization of conditions under which the observation is o 
be made, and, when possible, the use of electromechanical 
recording devices to ensure accuracy and completeness of 
coverage. 

Perhaps one of the more common types of intelligence 
activities that exemplifies controlled observation would be 
systematic propaganda analysis. 5 Under ideal conditions, the 
“unit” of the message to be noted and counted (e.g., a reference 
to an event or condition, a phrase, theme, or motif) would be 
identified preci sely before th e observation was made, the 

5 Not all propaganda analysis is performed “systematically.” In many 
instances the propaganda output is observed in an uncontrolled, ad hoc 
manner. 





















— ^ 25 

0\ y 15 

o P5 c 

~ 3 


0T 

ft ^ 

S % 

*-* rr 

_ P 

° C 
8 S' 

CD l<^ 
P P 

£ 3 

CL 

3 ►o' 

CD 3 
po O 
tn CfQ 

d ‘‘ 


O ^ P 
•-4 CD rf 

' CD rf O 
X CD 3 
£3 O on 

if'a 

CD 


X. C 

' C/5 3 


p °- 
q cd 

^ a 


cr cr 

CD rf 

3 2. O 

§ 5. £ 

3 g V) 

2 CTQ 0 
O g- P 

d 3 ‘ ' 

CD 
O 


►d 

»-*t 

CD 

C/5 m *H- 

CD 'X P 
3 0 3 
rf ^ Q- 

p ^ 


CD 

C/5 

3 

c 


p 


§f ° 

rf & CD 

£ 8 ^ 

2 o o 


d 

~ rf* OQ 

3 cf p 
C g 3 
o 5. O. 
cr _k. N*. 

d !3 
d dr go. 
. CD CTQ 

5^5 

CD ^ c /5 

p 2T cr 
0^3 

»-t < 

O P 

cr d. 


3T 

CD p 

^ C/5 

rf O 

cr T3 

CD T3 

-» o 

CD 
on 
CD 
P 

o o 

SJ* N-- 
CD d 


o 

d 


C o 


n g. 
3 ~ 

on ^ 


O 

05 

*-t 

CD 

< 

CD 


CW CT 

H c 3 
o Sg. 

3 8 2 
P rf *-J 

a s: „ 
as ? 


3d p. 3: 

2 CTQ 3 


CD p 

x nd 

U*2 3 b 

2 P era £- 


d CD 

o d 
o 05 


3 £ 


CD 
O 

cr 

d 

S 

cn d 
O CD 
p , on 

rf 

o p 

p-t 
CD 


1 rr5 

^ CD ^ 

cd 2 43 

d 3 

^ rf d 

O d* o 
^ td 
p ^ d. 

Sh-P 

3 $ _ 

1 

s g-8 

?§ £ 
0 P d 

o g* 2 
cr o 

C/5 d 

2 ? 
d p 3 


CD 

Q- 

O 

cr 

on 

CD 

3 

P 

rf 

o 

d 

on 

3 

p 

<< 

cr 

CD 

p 

CL 

CD 

c 

p 


O P 

•-* r-f 

<< CD 


CD O 

on 

CD 

P - 
d d 
O £ 
3* CD 


►-t ^ 

CD 5. 

£ ? 

rS ** 

CTQ 3 

rf 2 

O ^ 
►o - 


^ 2 

p ^ 

B- rf 

° •§ 

CD 

cr cl 

o o 
p d 

a* _ 

o dr 
p p 

on »-{ 
r-K Cl 
c« ^ 

o 

p-o 

§ •< 


w CD 

§ S 
I 3. 

d d 
p CTQ 

d rs 

o dr 

r-f CD 

3 o 

o g 

^ CD 

rr 

d 


p 


o 

d 


p 

CTQ „ _ 

? &3 

^OQ CL 

cd a h. 

CD* ^ d 


.. ^ -o « i-V 

P CL dr* CC 


O P 
■ 3 

CD c/5 
CL CD 

d 

O d 

5^ cr. ^ 

on O 
2 d 

^ n 


a 8 

>3 S' 


rv d P 

a | s- 

p d 0 

5* o 

cd cd 
CL ft) 

* O 

- pd £. 

CD fD O 

cr. o d 
o o ^ 
^ "d 
rT Cl , -t 
P t—• o 

"d d o 

CD CTQ CD 

On jyj 
05 


3 

p 

rrq 

d 


3 - 


CD 

q p on 

St cr 3 
d CD ^ 

d - O 

. L r+ 

O dr 

^ Cd CD 

SL « ^ 

cd ^ cr 

a a ° 


T1 . 
O 


o 3 ^ 
3. ° ° 

O r-K h-k 


CD 

d 


o 0 - 

g* §* ^ 

0 3 ? 

O P 

cr 

8 2 »■ 
3 3 S 

CD W 3 
cog 

h-. CD on 
on CL 

c 3 
Sow 
on r rr 
cd a 

d o _ 
<—f t—* o 

d* cr 
> n) r 
^ o o 

=r E ff 

SB'S. 

P r-f 

3* O §* 
C O on 
d d CD 
d r-f d 

P P < 

d d' P 

^ d r-f 

o 3' §’ 

CT 15 3 


•-$ d CD 


CD ^ 
>-t Xj 
05 p 

H 

P P 

3 


c 

05 

c 

p 


cr 

CD 

05 

rd 

2- 

cT 

CL 

o 

d 

r-f 

cr 

CD 

o’ 

Q 

r-f 

d - 

CD 

CL 

P 


df *“+ 

cd 

CD 


CD 

fi 

CD 

O 

o 

3" 

o 

r-f 

CD 

CL 


O P d* 

o o 3 
d o < 
add 

* r-f 

o* o o, 

B£ s . 

CL d 
d - 
d d: 

S" o* o 

” g 

^ c 

d - on 
d. CD 
O 

cr 


d o 

3 o 

CD d 

S o. 

5 5 

CD O 

05 ^ 


cl rs 

P CD 

sr ° 


d-*< 

° 3 

CD 
d 


3 

CD 

CD 


P CL 


i_K H 

cr u 

O CD 
on 

a* 

° 8 ? 

CD O 3 
l-f> p 

o cr. 
cr cl o 
g* p 3 

d. f-f on 
5 fo 

CD hQ 

cl p d 
d p 
< a c 

8 

Cl o &? 


CD p 

3 & 

rf 

£« 

CD _ 

P 3 

CD 

° 5 
^ o 

d CL 

O co 

rf 

S? 

cd ^y 


o H 

H-f) — 

CD CD 

3 p 

»-t d 
CD P 

3. d. 

3 2 

8 i 

3 O 
cr o 
3 ° 

on 

dC 05 

CD CD 


P 


o 

zr 


* o 


o C3 

■g o 
•-$ o 
g. d 

cr. d 
2 


CL 

o f 
g“o 

2 < 

3 ^ 

P CL 

rf 

O* 3 

d CD 
on on 

v» . rf 

O 5 2 

05" 

cd pr 

x 3* 

p d 

d 

>o a 
X- d 

CD CD 

‘ • 3 

d 

& CL 

d- o 

(P ° 


05 O 

CD po 

cT ^ 

a ° 

S. 2 

p 

d d. 
cl o 

<* 1:3 

d. on 

CL ~ 

d o 

p •-$ 

^ ht-j 

sr P 


> 

d 

O 

rf 

cr 

CD 


d O 


CD 

CL 

o 

cr 

on 

CD 

3 

p 

rf 

o’ 

d 

c 

d^ 

CL 

cr 

CD 

d 

o 


55’ > s 

CO d 3 

t=r .. cd 

2 >< SS 

| IS 

C CD 

d’^-xj 

t_— CD 

'S °g 

^ P CL 
CD d CL 

X o a* 

f— 1 cr cd 

“I s 
• § 8 
rr ^ 
5* cl 
2 CD 
d a 

CO _ 

a o 
cr -< 

2 ^ 
CL i-4 

d p 
d 

CD 05 
^ O 
*-i 


cd -a 05 
X CD o 
p d d 

3 o 3 

d' CL CD 
d on on 
CD 


cr 

CD 


o 

cr 


^ d 
o 3 
c g 
rr ° 

p w 
0-3 2 

<5 D* 3 

CD 

5* ^ CL 

St£ Ct 

g d CD 

a 2 - 

CD d 


2a 

^ a 
o ^ 

re o 

P »-{ 
CTQ _ 
P P 
d rf 

cl 2 

P ^ 
P 3 

3 p 


or 

CD 

S cr f 

d 3 

cl^ g 

rf ^ St 

cr 3 - p 

* oX 
o tr ^ 
o 


3- = 

» s 

3 o* 

CD CD 
D- on 

CD 

p 05 O 

3 sr SJ 

<D O ~- 

o. = a. 


o 

2" p 
CD 


^ d- 




BROADCASTING STATION: WARSAW I (RASZYN, POLSK.E RADIO 

FREQUENCY. 227 kHz 

MONITORING PERIODS: 0500-0730 l 2100-2300 Z 

DURATION: 7 SEP - 2 ' SEP 


WARSAW II iMOKOTONj POLSAIE RAOiO 

818 kHz 

0500-0730 Z 2100-2300 Z 

7 SEP - 21 SEP 

DAY MONITORED: 11 SEP 


REFERENCE TO OR COMMENTS 
ABOUT THE FOLLOWING 

VISIT OF SOVIET PARTY CHAIRMAN 
TO WARSAW 

PR0(+) ANTIC-) NEUT (0) 

NATO NAVAL EXERCISE TRIDENT 

PR0(+) ANTI(--) NEUT (0) 

FORTHCOMING •* EE7ING ®n T pnml!' S 
SECRETARY OF STATE AND POLISH 

foreign minister 
PR0(+) ANTI(—) NEUT (0) 
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(heir cooperation was essential, and that the questionnaire was 
,ulministered properly. But administering the questionnaire 
personally may also be expensive in terms of travel costs and 
lime. In addition, some respondents might hesitate to be candid 
when the researcher was present. 

When the researcher or some other responsible individual 
does not personally deliver and administer the questionnaire, 
(here is no assurance that the questionnaire will reach the 
desired respondents or that it will be administered properly, and 
this may be critical. 

Returns from “mail” questionnaires are generally low. (A 
“good” return from voluntary subjects may be as low as 20 
percent.) Consequently, if judgments are to be made on the 
basis of any statistical analysis of the responses, the initial 
sample of respondents must be fairly high. 

In addition, unless the questionnaire is reasonably short and 
is constructed in such a manner that it requires very little effort 
on the part of the respondent, a written questionnaire may be 
perceived as an unmitigated nuisance. Unfortunately, short, 
highly structured questionnaires can seldom address significant 
issues in the depth that the researcher needs or desires. 
Invariably, mail questionnaires require the researcher to make 
tradeoffs between superficial information from many subjects 
and in-depth information from a very limited number of 
subjects. 

In all probability, mail questionnaires, if they were to be 
used by the intelligence researcher, would probably be admin¬ 
istered to very small and highly select samples, as mentioned 
earlier. Coding devices can be placed on the questionnaires to 
enable the researcher to identify the respondents who com¬ 
pleted the questionnaires. Telephone calls, letters, or personal 
visits by the researcher could then be used to follow up the 
missing questionnaires. Sometimes cover letters signed by a 
high-ranking officer or by some other authority might motivate 
an uncommitted individual to respond. 

Questionnaire preparation is a laborious and time- 
consuming operation. Items must be written (“framed”), 
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number of cards received and the postmarks on the cards gave 
only a fair indication of the number of listeners but a good 
indication of the reception range of the broadcast station. 

Mail questionnaires are relatively inexpensive compared to 
other methods of collecting data. Large populations can be 
sampled in relatively short periods of time, trained interviewers 
may not be required, and the questionnaires can be completed 
at the convenience of the respondents. If standardization is very 
important (e.g., asking the same question in the same manner 
from all respondents), the questionnaire is most appropriate. 

On the other hand, as mentioned earlier, the number of 
returns from voluntary subjects is generally low, and even if the 
questionnaires were returned, there is no way of ensuring that 
they would be received in time to be used. Structured question¬ 
naire items invariably call for highly structured responses— 
responses which might not validly reflect the respondent’s true 
answer. Without the ability to “probe”, the researcher may 
never obtain information of the kind and quality that he needs. 
Furthermore, when the questionnaires are submitted 
anonymously, the researcher has no way of following up the 
responses. 

Perhaps one of the most serious drawbacks to the mail 
questionnaire is that the researcher has no control over the 
manner in which the questionnaire is administered. For 
example, as part of a larger study relating to drug use in the 
Army, highly confidential “drug use” questionnaires were to be 
given to all personnel in the grades E-l to E-5 in selected units. 
The questionnaires were to be completed by the subjects, 
placed in plain envelopes (which were provided), and dropped 
in a “ballot box.” In checking on the administration of the 
questionnaire in one unit, the researcher discovered that the 
unit’s first sergeant was administering the questionnaire orally 
to each subject. Obviously, the results obtained in this manner 
were completely useless, and the sample size was reduced 
significantly when the returns from this unit were discarded. 
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panel members for obtaining information about air defenses or 
damage, for example. 

Panels are especially effective for noting changes. But in 
order to use panels for this purpose, panel members must be 
queried more than once. For example, pilots would be 
debriefed after every successive mission flown against a specific 
target. 

Panels are “convened” when the observations, opinions, and 
judgments of the members are to be elicited. All panel members 
need not be present physically at one place and time in order 
for a panel to be “convened.” In many instances panel members 
might never meet their counterparts. Information could be 
elicited by use of a series of questionnaires sent out at 
appropriate times to each panel member, by telephone inter¬ 
views, or by interviews conducted personnally by the re¬ 
searcher. 

Panel techniques are effective for obtaining data for trend 
studies. Since information is obtained from the same subjects 
over time, any difference reported about the phenomenon 
observed is more likely to be a true difference rather than a 
difference arising from variance among subjects. 

The likelihood of getting the amount and quality of 
information needed from a panel is high since the members are 
handpicked by the researcher. In all probability, the members’ 
cooperation would have been obtained ahead of time, and the 
members would have to be briefed on the kinds of questions 
they would be asked. Another advantage of the panel technique 
is that any convenient form of eliciting information from the 
panel member can be used from written questionnaires to 
face-to-face contact. The major disadvantage of the panel 
technique is the potential for losing panel members through 
reassignment, sickness, or injury. 

Variations of the Critical 
Incident Technique 

The critical incident technique is a procedure used originally 
by industrial psychologists to isolate causal factors relating to a 
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specific; event-an accident for example. In applying tin 
mque researchers would examine all available documenl 
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Hl Procedures Marshall used were similar to procedures 
that other investigators might use, for example, General William 
Peers, investigating the My Lai incident. There are dTstinclionT 
owever. In Marshall’s approach, testimony could be reconciled 
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on the spot, and the cross-examination would not be incrimi¬ 
nating. In addition, emphasis would be placed upon establishing 
what had happened rather than who was responsible. The 
technique is also similar to debriefings, but again there are 
distinctions. The critical incident technique, and specifically 
Marshall’s variation of the technique, concentrates on a specific 
event, typically a unique, non-recurring type of event; whereas 
debriefings may address unique or recurring events. Debriefings 
are normally held to ascertain results. When critical incident 
techniques are used, the results are already known and it is the 
cause that is the focus of investigation. 

Instances in which the intelligence researcher might use the 
critical incident technique, or variations of the technique, 
include post operational analyses of the type Marshall per¬ 
formed, or they could be used with observers or participants 
directly or peripherally associated with an event, for example, 
participants of a reconnaissance mission. 

It should be noted that although the technique is classified 
as a data collection technique, the critical incident technique 
also involves analysis as well. For example, the technique 
involves cross-validating information, evaluating it, and placing 
the data in a context. These processes are essentially analytical. 
The technique is an eclectic one: it involves interviews, but it 
may also involve document analysis and firsthand observation 
on the part of the researcher as well. 

For More Information 

Useful references on field data collection techniques include 
the following: Mildred Parten, Surveys, Polls, and Samples: 
Practical Procedures Pauline V. Young, Scientific Social 
Surveys and Researchp William J. Goode and Paul K. Hatt, 


^Mildred Parten, Surveys, Polls, and Samples: Practical Procedures 
(New York: Cooper Square Publishers, Inc., 1965). 

9Young, Scientific Social Surveys. 
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Methods in Social Research} 0 and A. N. Oppenheim, Question 
naire Design and A ttitude Measurement. ^ 1 

■i If Ki the D reSea > rCher had to choose among the many works 
available Parten s Surveys, Polls, and Samples would be the besl 

choice. Jhe book is dated (although it has been reissued), but 
the basic tenets remain unchanged. Furthermore, the book nol 
only describes techniques, but it also contains step-by-step 
instructions for applying the techniques to the entire range of 
activities related to acquiring field data. B 

and H °^t\lr S ti en J if - C c° Cial Surveys and ^search and Goode 
and Hatt s Methods in Social Research are standard works in the 

ie , and both contain theoretical discussions and practical 
advice on the conduct of polls and surveys. Questionnaire 

usefrn inf Attltude Measurement by Oppenheim contains 
seful information on the mechanics of constructing question- 
aires, interview schedules, and rating scales, as well as 
information on the administration of these devices. 

r„/^ n ? the p W ° rk related to obtai ning field data is A 
Guide for Field Research in Support of Psychological Opera¬ 
tions by Alexander Askenasy and Richard H. Orth. 12 Although 
the guide is directed toward PSYOP activities in foreign 
countries, the principles of interviewing, questionnaire prepara¬ 
tion and administration, and observation are relevant to field 
data collection anywhere. 


Go , ode and Paul K- Hatt, Methods in Social Research 
( ork. McGraw-Hill Book Company, Inc., 1952). 

(New ^rlc B^c^ookT.'lnc.^r^dX^ and Attitude Measurement 

12 Alexander Askenasy and Richard H. Orth, A Guide for Field 
esearch in Support of Psychological Operations (Kensington, Maryland 
American Institutes for Research, April 1970, AD 869-057L) Y 
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A masterpiece (and perhaps the only one of its kind) is 
Eugene J. Webb et al, Unobtrusive Measures: Nonreactive 
Research in the Social Sciences. The book describes innovative 
techniques for obtaining research data under conditions in 
which the presence of the researcher would interfere with the 
quality of the data collected, or under conditions in which it 
would be impossible for the researcher to collect data by 
interacting with the subjects. Despite the formidable title, the 
work is very readable and is most appropriate for the 
intelligence researcher, the analyst, the attache—in fact, anyone 
who might have to collect data unobtrusively. 

Summary 

[> Libraries do not always have the information or data a 
researcher needs. When the researcher collects information 
directly from the source he employs one or more field data 
collection techniques. 

>Interviews are common field data collection techniques. 
Interviews may be highly structured or they may be unstruc¬ 
tured or “open-ended.” They may be conducted individually 
with one subject at a time, or they may be conducted in a group 
setting. Interviews might be expensive and time-consuming. But 
they permit the researcher to follow up leads immediately. 

D>It is generally cheaper to administer questionnaires than it is 
to conduct interviews. Well-constructed questionnaires elicit the 
same kind of information in the same manner from all 
respondents. Thus, it is generally easier to analyze questionnaire 
data than it is to analyze transcripts of interviews. But 
questionnaires may take a long time to develop, test, and 
validate. Furthermore, unless the respondent is required to 
reply, the response rate on questionnaires is notoriously low. 

t> Observation methods are considered to be controlled or 
uncontrolled. In controlled procedures, the researcher records 
information about the same types of events in a regular, 
systematic pattern. In uncontrolled observations, the researcher 
records what is significant, if and when it occurs. 
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combinations of techniques am r re ^ ui ^ ements - More often, 
ing up a mail questionnaire ?“ lred; for example: follow- 
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CHAPTER XIV 

FOUNDATIONS OF ANALYSIS: 
SOME BASIC CONCEPTS 


It is not enough to possess the needed relevant knowl¬ 
edge in some dormant state; we have to recall it when 
needed, revive it, mobilize it, and make it available for 
our problem, organize it. 

George Polya 

Data collected in the preceding phase are “mobilized” in the 
analysis phase of a research program. This phase constitutes the 
“moment of truth” of any research project. It is in this phase 
that assumptions, hypotheses, and guesses are put to the test. It 
is in this phase that interpretations, significance, and meaning 
are ascribed to the data. 

This chapter discusses basic concepts that underlie all types 
of analytic processes, regardless of whether the processes are 
performed by an intelligence analyst or performed in the larger 
context of intelligence research. Topics covered in this chapter 
include rules for verification, causality and correlation, canons 
(or rules) for establishing causality, and the sometimes nebulous 
distinction between “quantitative” and “qualitative” ap¬ 
proaches to analysis. 

Analysis 

Analysis has been defined as the process of breaking down a 
problem into its component parts and studying the elements 
separately; 1 as the mental process of “handling of data by the 
analyst for the purpose of incorporating the data . . . into the 


1 Smith and Smith, An Introduction to Research in Education , p. 185. 
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ext of a report ; and as the “ . . . minute examination of 
related items to determine the extent to which they confirm 
supplement, or contradict each other and thereby to establish 
accepted facts and relationships..” 2 3 

M There^ are subtle distinctions between the use of the term 
analysis” in intelligence production and the use of the term in 
research, but essentially the term connotes breaking down larger 
components or elements of a problem into smaller units, 
performing mental (and sometimes physical) operations on 
co ected data, and, on the basis of the operations 
performed, arriving at a generalization or conclusion about the 
larger problem that is being addressed. 4 Interestingly, the 
term “analysis” is derived from the Greek words ana, which 
means up,” and lyein, which means “to loosen”: hence, “to 
loosen up the data, a most appropriate description of the 
operation. 

Induction and Deduction: 

A Short Review 

It will be recalled from the earlier chapters that deductive 
reasoning is the act of drawing conclusions from previously 
formulated premises. It is the type of reasoning that is used, for 


2 De f en se Intelligence School, “The Analytic Process” (Washington 
D.C.: The Defense Intelligence School, 1965). 

3 Armed Forces Staff College, “Intelligence for Joint Forces” (Nor¬ 
folk, Virginia: Armed Forces Staff College, 1966). 

4 ln the technical terminology of intelligence, analysis is usually the 

term that is applied to all activities and processes performed on collected 
data, including the formulation of hypotheses, the determination of the 
relevance and validity of the data, and collation and classification. In 
conventional research terminology, some of these activities would be 
performed in different phases of a research program. For example 
hypotheses would be formulated during the problem definition phase, and 
relevance of data would be addressed during the data collection phase. 
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example, when mathematical proofs are established. And when 
used in closed systems such as mathematics, deduction is 
“. . . safe, beyond controversy, and final.” 5 Deduction is 
sometimes referred to as demonstrative reasoning because it is 
used to demonstrate the truth and validity of a conclusion, 
given certain premises. 

In contrast to deduction is induction. Induction, it will be 
recalled, is the process of arriving at a generalization on the 
basis of one or more observations. Unlike deduction which 
simply makes clear what is already known, induction provides 
new knowledge. Furthermore, induction or plausible reasoning 
invariably precedes deduction and it is the type of reasoning 
that analysts and researchers are required to perform more 
frequently. 


5 George Polya, Mathematics and Plausible Reasoning , vol. II: Patterns 
of Plausible Inference (Princeton, New Jersey: Princeton University Press, 
1954), p. v. 

George Polya, the Stanford University mathematician, is perhaps the 
lending contemporary advocate of “plausible reasoning.” In five separate 
but related works, Polya attempts to refine the science (or art) of problem 
solving or “heuristics,” as it is formally called. Polya is quick to point out 
that it would be naive to believe that any given set of rules of problem 
solving could be applied universally to all kinds of problems. He does make 
an excellent case, however, for developing rules of discovery and 
verification in his own professional field of mathematics. For the greater 
part, most of the examples Polya uses to illustrate his “rules” pertain to 
mathematics. But it takes very little imagination to see how his “rules” 
would apply to other domains as well; for example, to intelligence, 
propaganda analysis, or law. 

Some years ago HRB-Singer researchers discussed with Polya the 
relevancy of “heuristics” to intelligence analysis. Polya’s advice was 
essentially that heuristics had to be developed specifically for each 
separate domain. The “rules” of verification that are suggested in the 
following pages are attempts to begin structuring an heuristic approach for 
intelligence analysis. The contribution of Polya is obvious. 
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Induction involves the processes of discovery and verifica 
tion. With respect to the act of discovery, general rules or 
guidelines are nonexistent. For example, Polya states, “The first 
rule of discovery is to have brains and good luck. The second 
rule of discovery is to sit tight and wait until you get a bright 
idea —hardly comforting admonitions to the analyst or 
researcher facing a deadline. 

With respect to verification, the second part of the inductive 
process, however, there are certain guidelines which the 
researcher may find useful. Induction can never yield “truth” 
with the rigor that deduction can demonstrate. Induction, on 
the other hand, can yield highly plausible conclusions which can 
be tested over time. 

Since verification constitutes an important part of the 
analytic process, general “rules” for performing verification are 
discussed below. These general rules for verification apply 
equally to the use of qualitative or quantitative data as well as 
to every type of analysis in which an analyst or researcher is 
required to arrive at some conclusion. 

Seven General Rules for 
Verification 

The product of the process of induction is a generalization. 
Having arrived at a generalization (guess, conjecture, hypoth¬ 
esis), the researcher’s problem changes from one of discovery to 
one of verification. Described below are seven general rules for 
determining the plausibility of a generalization. Note: the rules 
are methods for establishing likelihood, not certainty , that a 
generalization is true. The general rules are paraphrases of 
Polya’s rules for plausible reasoning. 6 7 


6 George Polya, How to Solve It (Garden City, New York: Doubleday 
& Company, Anchor Books, 1957), p. 172. 

7 Polya, Patterns of Plausible Inference, p. 5. 
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1) A generalization is more believable when a consequence 
<>f that generalization is verified. 

I or example, an analyst might generalize on the basis of 
earlier observations and information that a new missile that was 
being developed by a foreign country was nearing an opera- 
lional stage. This generalization would be strengthened should 
photographs of a test facility reveal burn marks on the concrete 
apron where a missile’s propulsion system was tested. Clearly, 
(he burn marks do not prove that a missile’s propulsion system 
was working well, but the marks are not inconsistent with a 
successful test on the pad. 

2) The credibility of a generalization increases as the 
different means used to test the generalization support it. 

To return to the operational status of a missile, should a 
country declare the airspace near its missile test range off limits 
lor a certain period, the initial generalization of the researcher 
or analyst that the missile was nearing operational status would 
increase. The observation of increased activity at the test range 
would further tend to support the initial generalization, as 
would increased communication traffic on the nets used for test 
purposes. 

3) Confidence in a generalization increases as the observa¬ 
ble bits of evidence which support the generalization bear some 
proximity to each other. 

Proximity can be expressed in terms of time or geographic 
location. In the example above, increased activity at a test range 
and the declaration that the airspace over the test range was 
restricted bore both chronological and geographical proximity 
to each other. 

4) The credibility of a generalization is directly propor¬ 
tional to the number of instances in which the generalization 
was supported. 

Stated somewhat differently, every time a generalization is 
supported by new bits of evidence, the strength (plausibility) of 
that generalization increases. 
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5) Confidence in a generalization increases when an imcom 
patible and rival conjecture is refuted . 

In the example of the missile test above, the bum marks 01 
the concrete apron could have been caused by an accidental 
explosion. However, if the test facilities around the test stanc 
showed no indication of damage, this hypothesis would be 
discounted and the original conjecture would be strengthened 

6) Confidence in a generalization increases to the extent 
that it is consistent with another generalization that is highly 
credible. 

For example, if the country in which the status of the 
missile was in question invariably followed the same sequence 
of test and development phases that another country followed 
about which more information was available, the researcher or 
analyst would tend to have more confidence in predicting the 
next event in the missile’s development cycle. This is tanta¬ 
mount to reasoning by analogy, and although this is a fallacious 
type of reasoning in logic , it is a very practical and common 
procedure in intelligence research and analysis. (This will be 
discussed further in Chapter XVII.) 

7) In instances in which observables support two different 
generalizations , the simpler generalization stands a better 
chance of being true. 

In the missile test example cited earlier, it would be 
conceivable that the missile’s propulsion system had failed 
during the tests, but that in order to convey the impression that 
development was progressing according to schedule, spurious 
positive indications had been generated intentionally. This is 
certainly not an impossibility, but the simpler explanation that 
the test had been successful intuitively appears more plausible. 

Causality and Correlation 

A fundamental concern in intelligence research and analysis 
is the identification of causal factors. 8 Intelligence research is 

8 Nor is it surprising that this is the focus of much basic scientific 
research—particularly medical research. 
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concerned with identifying causal factors because these factors 
IH'rmit one to predict or to anticipate results and effects. 

In certain types of intelligence analysis, causal factors can 
lu identified readily, and the effects of these causal factors can 
In- predicted accurately. Not surprisingly, these are the areas o 
, ientific and technical intelligence. For example, the range ot a 
missile is proportional to the velocity of the missile at burnout. 

I hus given the velocity of a new missile at burnout, the range 
id (hat missile can be estimated. 9 

In the “softer” types of intelligence, analysts are not so 
fortunate. Not only cannot the effects of certain variables be 
predicted, but there may also be little agreement as to what 
constitutes the causal or contributing factors in the first place. 

II is because of this basic inability to establish causes rigorously 
lhat intelligence is (and probably always will be) referred to as 
an art or a craft rather than a science. 

The case for causality is not hopeless, of course, and there 
are many instances in which at least a plausible if not a 
definitive case can be made for stating that a certain event, 
condition, or phenomenon brought about another phenom¬ 
enon, or that a given event will bring about a certain type of 
response or reaction. Because this function is so basic to 
intelligence research and analysis, attention will be paid to 
procedures by which causal factors can be determined. 

Perhaps one of the most common methods for establishing 
causality is to consider initially those events or conditions 
which are correlated. Correlations may be said to exist when 
two or more phenomena occur or vary in relation to each other. 
Obviously, because two phenomena occur or vary together does 
not imply that one phenomenon is the cause of the other. For 
example, there may be a positive correlation between height 
and weight, but obviously one trait is not the cause of the other 
because some people are tall and light and others may be short 
and heavy. But the ability to establish correlations is often a 
first step in establishing causes, and established correlations 
permit one to generalize from characteristics of a known 

9 0f course, the missile could also be tracked by radar and its range 
determined directly. 
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phenomenon to characteristics of an unknown (but correlated) 
phenomenon. 

Correlations are used very often in current intelligence and 
m estimative/predictive intelligence analysis. “Indicators”, for 
example, are signs or manifestations that are associated (corre¬ 
lated) ^ with certain events. Certain events or conditions are 
called precursors” because they had been correlated with other 
events which occurred later. Certain precursors are not only 
correlated factors, but they may also be causal factors. 

When two events, conditions, or characteristics vary to¬ 
gether and in the same “direction” (e.g., an increase in weight 
correlated with an increase in size), a positive correlation is said 
to exist. When two events vary inversely to each other (e.g., an 
increase in size with a decrease in speed), a negative correlation 
is said to exist. From the standpoint of inferring an unknown 
characteristic on the basis of a known and correlated character¬ 
istic, it does not matter which type of correlation exists. What is 
important, of course, is to understand 1) that two phenomena 
are directly related, and 2) to understand the manner (nega¬ 
tively or positively) in which they are related. 10 Perhaps this 
point is obvious, but the authors have encountered many 
researchers who expressed dismay when their tests revealed 
negative correlations. Yet, from the standpoint of predicting or 
inferring from a known to an unknown, a negative correlation 
may be as useful as a positive correlation. 

As an aid to analysis, the establishment or identification of 
correlated events, activities, conditions, or other phenomena 
may permit other inferences to be drawn, often with a high 


10 Also important, of course, is the “strength” of the correlation, i.e., 
how well or to what degree two phenomena are correlated. The strength of 
a correlation is partly a function of the sample size. Unfortunately, in 
intelligence research and analysis, sample sizes tend to be very small. 
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degree of predictability. And, as will be discussed below, the 
identification of correlates is basic to the identification of 
causal factors. 

Canons of Causality 

The ability to establish causality is essential to valid 
prediction. As W. Stanley Jevons put it, 

A cause is defined as the necessary or invariable 
antecedent of an event, so that when the cause exists the 
effect will also exist or will soon follow. If then we 
know the cause of an event, we know what will certainly 
happen . . . - 11 

Although intuitively the researcher may feel that certain 
factors “caused” or brought about a certain result, it is essential 
that at some point in his analysis he replaces his intuition with 
stronger, more “public” evidence. In short, it is necessary that 
he subjects his speculations to certain tests. One set of tests of 
causality are the rules set forth by the Utilitarian philosopher, 
John Stuart Mill. The rules are commonly referred to as “Mill’s 
Canons.” They epitomize the scientific method (most experi¬ 
mental designs incorporate them in one way or another), and 
although they are not without their detractors (Jevons being 
one), they are most relevant to intelligence analysis. The 
“ canons ” are paraphrased below. 

1) If two or more instances of a phenomenon have only 
one characteristic or feature in common, the single character¬ 
istic or feature that exists in all instances is the cause (or 
effect) of the given phenomenon. 

Applied to intelligence research or analysis, this canon 
might be applicable in instances in which, for example, the 
presence of a Soviet missile expert had been noted at various 
test sites prior to the firing of a new missile. Diagrammed, this 
canon would look like this (Figure XIV-1). 


11 W. Stanley Jevons, The Principles of Science: A Treatise on Logic 
and Scientific Method (New York: Dover Publications, Inc., 1958), p. 222. 
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A 1 ' 2 ' 3 - DIFFERENT INSTANCES OF THE SAME TYPE OF PHENOMENON (e,g MISSILE TEST FIRINGS ) 

.e., THE CAUSE, 




THE SINGLE FEATURE COMMON TO ALL OCCURRENCES OF THE PHENOMENON, 
AN IMPORTANT PART OF THE CAUSE, OR THE EFFECT OF THE PHENOMENA 


Figure XIV-1. Mill’s First Canon 



THE PHENOMENON WITH ITS ASSOCIATED 
CHARACTER I ST ICS,FEATURES, OR 
CIRCUMSTANCES eg, THE PRESENCE OF 
A MISSILE EXPERT AT A TEST SITE, AND 
THE FIRING OF A MISSILE, 


AN INSTANCE IN WHICH A 
PHENOMENON DID NOT OCCUR 
(NO MISSILE WAS FIRED), BUT 
IN WHICH ALL CONDITIONS EXCEPT 
THE PRESENCE OF THE MISSILE 
EXPERT EXISTED 


THE SINGLE FEATURE OR 
CIRCUMSTANCE THAT IS 
THE CAUSE,AN IMPORTANT 
PART OF THE CAUSE, OR 
THE EFFECT OF THE 
PHENOMENON,, 


00 

UJ 


Figure XIV-2. Mill’s Second Canon 




































2) If an instance in which a phenomenon occurs , and an 
instance in which a phenomenon does not occur , have every 
characteristic or feature all common but one , and that one 
feature was present only when the phenomenon occurred , that 
single characteristic (feature or circumstance) is the cause , an 
important part of the cause , or the effect of the phenomenon. 
For example, in the case of the missile expert’s association with 
test firings, this canon would apply when all conditions existed 
for a test firing except the presence of the missile expert (and, 
of course, the non-occurrence of a missile firing). (Figure 
XIV-2.) 

3) If two or more instances in which the phenomenon 
occur have only one feature , condition , or circumstance in 
common , while other instances in which the phenomenon does 
not occur have nothing in common except the absence of that 
single condition , the condition that sets the instances apart is 
the cause , an important part of the cause , or the effect of the 
phenomenon. For example, if two instances were encountered 
in which the presence of a new radar could be associated with 
exceptionally accurate air defenses, and several instances were 
encountered in which these radars were not detected, and in 
which the air defenses were no more accurate than usual, the 
cause of the increased accuracy of the air defenses could be 
attributable to the newly detected radars (Figure XIV-3). 

4) For any phenomenon , list all of the pertinent antecedent 
conditions. Then determine which parts of the phenomenon 
were caused by specific antecedents and subtract these parts 
from the overall phenomenon. The remaining parts of the 
phenomenon are the effects of the remaining antecedents. 

This canon is most appropriate for laboratory situations or 
in situations in which all pertinent antecedent conditions can be 
identified. But the identification of all relevant antecedent 
conditions may be extremely difficult in unstructured situ¬ 
ations. As tenuous as this canon may appear, in reality it is 
applied quite often. For example, in scientific and technical 
intelligence analysis, the components and functions of a piece 
of captured equipment would be identified. Control devices 
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Figure XIV-3. Mill’s Third Canon 


























would be associated with various functions. Functions for 
which control devices would not be identified initially would be 
controlled by the remaining devices not related to other 
functions. In layman’s terms, this activity is referred to as the 
“process of elimination” (Figure XIV-4). 

5) When a phenomenon varies whenever another phenom¬ 
enon varies , the first phenomenon is a cause of , the effect of, or 
is connected to the second phenomenon. (This is a succinct 
statement of the relationship of correlation to causality 
discussed in the previous section.) For example, certain un¬ 
known telemetry signals might be associated with certain 
maneuvers of a missile that was being tracked by friendly 
radars. The extent to which the signals changed invariantly with 
maneuvers of the missile would be the extent to which those 
signals would be considered the cause of the missile’s maneuvers 
(Figure XIV-5). 

Obviously, causes always exist or occur before effects. 
Admittedly, the difference in time might be measurable in only 
nanoseconds, but “A” must precede “B” if “A” is to be 
identified as the cause of “B.” Finally, it should be apparent 
that causal factors are always correlated factors, but not all 
correlated factors are causal factors. 

"Quantitative" and "Qualitative" 

Analysis: A Nebulous Distinction 

It is a truism to say that analysts and researchers are 
concerned with the rigor of their investigation. But what may 
not be so obvious is that one of the ways by which researchers 
attempt to attain rigor (or at least give the impression of rigor) 
may be fallacious. Specifically, naive researchers assume that 
the presentation of information in “quantitative” terms some¬ 
how enhances the quality of the information, or increases the 
reliability and validity of the conclusions. This is erroneous, of 
course, but the mystique of numbers persists. Casting crude 
observations into some statistical form will not improve the 
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Figure XIV-4. Mill’s Fourth Canon 





































quality of the research, nor will highly precise quantification of 
badly classified information yield anything more substantive 
than what there was initially. 12 

The phenomenon of “false precision” is not unique to 
intelligence research or analysis, of course. In a review of a book 
dealing with American intellectuals, Paul Starr, a Yale sociolo¬ 
gist, made the following observations: 

-can wax enthusiastic over the most trivial 

statistic. Influential nonacademics, he tells us at one 
point, claim to read regularly “a whopping 16.5 journ¬ 
als.” The less influential read a less whopping nine. The 
copy here seems to have been misplaced from an 
advertising agency. 

In the middle section of the book, devoted to the 
intellectuals’ response to Vietnam,-catalogues and 


1 Classified in this context means being assigned to a category. 
Classification is discussed in greater detail in Chapter XV. 
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totals up opinions with the thoroughness and perspi¬ 
cacity of a bookkeeper.... 

Examining the sources of opinion on the war,- 

comes up with the following exact statistics: 20 percent 
were influenced by The New York Times, 16 percent by 
leading intellectual journals, 10 percent by television, 31 
percent by direct experiences and 13 percent by “diffuse 
personal encounters”—all these precise figures based on 
interviews conducted in 1970 on what influenced 
opinions 5 to 10 years earlier. 

This kind of false precision is a parody of social 
science. Science does not call for the trivializing of ideas, 
or for data that are little more than artifacts of the 

methodology. 1 *^ 

A misunderstanding of the application of statistics to any 
type of studious inquiry is reflected in the persisting dichotomy 
between qualitative and quantitative research or between 
nonstatistical and statistical approaches. But qualitative and 
quantitative approaches are merely logical extensions of the 
other. For example, that which is measured or quantified is 
always a quality. Quantification is a method of expressing a 
quality with more precision //—and this is critical— if that 


12 Paul Starr, a review of The American Intellectual in The New York 
Times Book Review, September 15, 1974. 
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quality is defined with sufficient clarity to permit reliable 
measurement.! 4 

By the same token, there is implied measurement in many 
of the concepts which would normally be considered “qualita 
tive. For example, to say that one government is more 
belligerent than another implies a comparison of degrees of 
belligerency, a crude form of measurement. A political scientist 
who comments upon the high degree of political cohesiveness 
within a certain country is making a quantitative statement 
about a quality. When analysts or researchers couch their 
degrees of certainty about an event’s occurring in terms of 
“ . . . it is almost certain that,” or “ ... it is highly probable (or 
improbable) that. . . ,” they are making qualitative statements 
which are in fact quantifiable in terms of probabilities. The 
Sherman Kent Chart (Figure XIV-6) is a device for converting 
qualities of intelligence statements into quantitative expressions 
of likelihood. 

The distinction between approaches becomes even more 
tenuous when the various scales used for “measurement” are 
considered. The nominal scale, for example, is essentially a 
nonquantitative scale for assigning elements to various cate¬ 
gories, i.e., classifying data, a topic discussed in Chapter XV; 


^Reliability refers to the extent to which successive measurements of 
the same phenomenon would yield the same results. 
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CHANCES 


FOR - AGAINST 


100 - 0 

CERTAINTY NO ESTIMATE 

99 - 1 

ALMOST CERTAIN. HIGHLY L1KELY. 

85 - 15 


84 - 16 

PROBABLE. LIKELY. PROBABLY. WE BELIEVE......... 

55 - 45 


51 - 49 

CHANCES JUST BETTER THAN EVEN. ON BALANCE,, ..... 

50 - 50 

CHANCES ARE EVEN. 

49 ~ 51 

CHANCES JUST LESS THAN EVEN. 

45 - 55 


16 - 84 

IT IS DOUBTFUL. WE DOUBT. IMPROBABLE. UNLIKELY. 
PROBABLY NOT. 

15 - 85 

ALMOST CERTAINLY NOT. HIGHLY UNLIKELY. 

CHANCES ARE SLIGHT. 

* 1 - 99 


0 - 100 

IMPOSSIBILITY NO ESTIMATE 

Figure XIV-6. The Sherman Kent Chart, a Device for 


Converting Qualitative Judgments into 
Quantitative Terms (and Vice Versa) 
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and ordinal scales are “quantitative” only in the sense that the 
elements measured are rank-ordered in terms of magnitude. 

The important concern in any type of analysis is precision, 
but analysis cannot increase the precision of data that were 
collected “imprecisely,” or of data that are poorly defined. 

Summary 

t>The term “analysis” has slightly different connotations in 
intelligence research and in academic research. Despite the 
subtle distinctions, analysis, in both contexts, refers to breaking 
down a large problem into a number of smaller problems and 
performing mental (and sometimes physical) operations on the 
data in order to arrive at a conclusion or generaliza¬ 
tion. 


l^Four types of scales are used in measurement: nominal, ordinal, 
interval, and ratio scales. The scales are defined as follows: 

1) Nominal Scale: This scale categorizes qualities. Applied to 
vehicles, for example, vehicles could be classified, or categorized, {or 
measured)by their type of propulsion, e.g., gasoline powered or diesel 
powered; or aircraft, by the nature of their wings, e.g., fixed, rotating, or 
swing. Nominal scales are the types of measurement scales used in 
classification, a basic step in the analytic process. 

2) Ordinal Scale: This scale categorizes (or classifies) on the basis 
of qualities that can be rank-ordered; e.g., a tracked amphibious vehicle 
could be classified as being “more mobile” than a wheeled, non- 
amphibious vehicle. Foreign countries might be rank-ordered in terms of 
their strategic importance to U.S. national interests. 

3) Interval Scale: This scale has the characteristics of the ordinal 
scale in addition to numerically equal distances between points on a scale. 
Fahrenheit and centigrade thermometers are examples of devices using 
interval scales. (The zero point on interval scales is arbitrary.) 

4) Ratio Scale: This scale contains an absolute or true zero point 
in addition to equal units. Examples would be measures of weight, length, 
and height. 
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D> Deduction and induction are two basic mental operations 
performed on data. Deduction is referred to as demonstrative 
reasoning because it is the process used to demonstrate the 
truth and validity of a conclusion. Induction is sometimes 
referred to as plausible reasoning. It is the type of reasoning one 
employs in discovering and verifying generalizations. It is also 
the process used most commonly in problem solving. 

t> Although formal rules exist for deduction, no formal rules 
exist for induction. Nevertheless, certain generalizations can be 
stated in terms of conditions which increase the plausibility of a 
generalization’s being true. These generalizations were stated as 
rules for verification or rules of plausible reasoning. 

[> The ability to establish the cause of an effect is basic to 
predicting outcomes. Therefore, scientists and intelligence 
researchers are very much concerned with establishing cause and 
effect relationships. 

|> A correlation always exists between causes and effects, but 
not all correlations are causal relationships. 

t> Commonly used rules for establishing causal relationships 
are John Stuart Mill’s “canons” which were described in this 
chapter. 

> Distinctions between quantitative analysis and qualitative 
analysis become tenuous when it is recognized that what is 
quantified is always a quality, and that certain distinctions 
based on qualitative features often reflect a basic form ot 
measurement or quantification. 

t> Simply because a phenomenon was expressed in quantita¬ 
tive terms does not necessarily imply that the research was 
“rigorous.” Even the most rigorous mathematical or statistical 
analysis cannot improve the quality of poor data. 
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CHAPTER XV 

CLASSIFICATION: A BASIC STEP IN ANALYSIS 


Classification is the ‘halfway house’ on the road to 
measurement. 

Alfred North Whitehead 

Classification is the process of assigning information to 
classes. 1 The process is indispensable in research projects 
involving large-scale surveys where many cross-tabulations must 
be performed, and it is often useful in small-scale research 
studies, not only as a means for expediting data handling, but 
also as a means of problem solving as well. 

Two examples cited in this chapter deal with classification 
in the biological and physical sciences. Significant discoveries 
have been made in the sciences as a result of the process of 
classification and, conceivably, similar “discoveries” can also be 
made in intelligence. 

Classification in Science and 

Intelligence 

Scientists who are able to assign an object, an organism, an 
event, a condition, or an activity to a class are often able to 
infer qualities about the phenomenon that would not be 
apparent otherwise. 

For example, a zoologist may have noticed a track of an 
animal with a four-padded foot and retractable claws. With his 
knowledge of zoological categories, the zoologist would assign 


^In intelligence research and analysis, data are often “classified”, i.e., 
assigned to specific categories, on the basis of the degree of danger to 
national security that would result from their unauthorized disclosure. 
This is perhaps the most familiar sense in which the term is used in 
intelligence. 
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this animal to the class of mammals. (Amphibians have four 
toes, no claws; reptiles, five toes but nonretractable claws.) 
Having assigned the unobserved animal to the class of mammals, 
the zoologist could infer other characteristics about the unseen 
animal based on class characteristics, viz., that it is a hairy 
animal and that it nurses its young. The zoologist could infer 
that since the animal had retractable claws, it ate meat As a 
meat eater, the animal most likely had incisors, canine teeth, 
and probably molars. Thus, on the basis of one footprint a 
zoologist could reconstruct a fairly accurate picture of t e 
unseen animal simply by assigning the obse^ed phenomenonto 
a class and then inferring other characteristics that all members 
of the class possessed. 

The intelligence counterpart of this analog can be seenAn 
the analysis of a foreign aircraft’s capabilities. For example, the 
Russian Sukhoi attack aircraft, “Fencer,” aside from its vanab■ e 
geometry, bears superficial resemblance to the Su-7 
ter.” Although close examination of both aircraft reveals 
distinguishing features, one can assume that the missions 5 
probably comparable, and that the performance of the later 
“Fencer” would not be any lower than its predecessors, the 
Su-7 and the Su-17 aircraft. 

Similarly, the 11-18 medium-range transport is a “civilian” 
aircraft that is almost identical to the 11-38 ASW/patrol aircra . 
Since civilian aircraft are generally more easily observed tha 
are military aircraft, a critical study of this aircraft would 
suggest characteristics of the less accessible military version. 

To show further how classification can be used m intelli¬ 
gence research or analysis, the evolution of the periodic table 
will be discussed briefly. In 1869 Dmitri Ivanovich Mendeleev 
published a paper in which he classified the elements according 
to the properties the elements manifested. He wrote. 

When I arranged the elements according to the magni¬ 
tude of their atomic weights, beginning with the 
smallest, it became evident there exists a kind of 
periodicity in their properties... I designated the name 
“periodic law” to the mutual relation between the 
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properties of the elements and their atomic weights. 

These relations are applicable to all the elements, and 
have the nature of a periodic function.2 

Significant is the fact that in certain instances he found that 
his criterion for classification (the atomic weight) was inappro¬ 
priate for ordering specific elements. For example, had 
Mendeleev adhered only to atomic weights, tellerium, with an 
atomic weight of 127.61, would have come after iodine 
(126.91). However, on the basis of its other properties, placing 
tellerium ahead of iodine on a “periodic table” put it under 
selenium which it resembled closely, and iodine then fell under 
bromine which it also resembled closely. 

The value of this scheme became apparent when Mendeleev, 
not being able to complete his table, announced that the 
elements that should fit the empty spaces on his table “must be 
found.” Furthermore, on the bases of the properties of the 
elements that preceded and followed those spaces, Mendeleev 
predicted what the properties would be of those elements that 
still had to be discovered. (Within his lifetime three of his 
predicted elements were discovered, and the elements had those 
properties predicted by Mendeleev.) 

Creating an order of battle is an intelligence counterpart of 
constructing a periodic table. For example, based on a 
knowledge of enemy doctrine and from past experience, it 
might be established that the enemy conventionally employed 
two units on line while holding one unit in reserve. On the basis 
of the identity of one unit on line and another unit, presumably 
in reserve, and on the basis of the composition of both known 
units, the composition and possibly the identity of the other 
unit on line could be inferred tentatively, even though contact 
with this unit had not yet been made. Of course the intelligence 
analyst Oust like Mendeleev) does not know for certain what 
the composition of the missing “element” will be. But he does 
have a plausible idea. 


2 Harold A. Larrabee, Reliable Knowledge (Boston: Houghton Mifflin 
Co., 1945), pp. 239-248. 
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In another example of how classification permits analysts 
and researchers to make inferences, an analyst might know that 
the enemy conventionally deployed a certain weapon system 
eg a SAM complex, in a particular geometric pattern. It the 
analyst received partial information relating to an unknown 
weapon system which fitted the deployment pattern of the 
weapon system known previously to the analyst the configura¬ 
tion of this pattern would permit the analyst to assign the 
weapon system to a particular group of weapons. From a 
knowledge of the group’s characteristics, the analyst could lnfe 
the configuration of the remainder of the deployment patte 
as well as the probable characteristics of that system. 

Systematic propaganda analysis, or content analysis (of 
foreign publications, mass media, or public announcements by 
key military or political figures) is another intelligence applica¬ 
tion of classification. In its most basic form systematic content 
analysis involves the examination of public utterances, the 
identification of content elements of interest, the assignment of 
coded utterances (content elements) to predetermined cate¬ 
gories, and the analysis of the results of the tabulations (e.g., 
correlations, interpretations, and so on). Although simp y 
described, the implementation of the procedures may 
considerably more difficult. For example, the content element 
(or better, the recording unit) may be a phrase, several words 
or a single term. 3 4 Thus, recording units must be defined 
precisely and so must the classes or categories to which the 
recording units are to be assigned also be defined unambig¬ 
uously. Rules for assigning recording units to classes y 
require thesauri that are hundreds of pages long. 


3 Bernard Berelson, Content Analysis in Communication Research 
(Glencoe, Illinois: The Free Press, 1952), page 135. 

4 What really confounds the coding and classification process in 
propaganda analysis is that the statements being classified may themselves 
be ambLous. For a discussion of additional problems in propaganda 
analysis see Jerome K. Clause,, Propaganda 

Procedures for Analysis (Stale College, Pennsylvania, HRB-Smger, 1973). 
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Why Classify 


Analysts and researchers classify for two main reasons- to 

understand their data, and for convenience. As an aid to 

understanding classifications often reveal relationships that are 

ot perceptible among unorganized data. As relationships are 

revealed, often additional bases for comparison and prediction 

can be established. To go back to two examples previously 

wa^aH W f by C l 3SSlfying and ordering his data that Mendelee^ 

Similarly an *2 TV pr ° perties of the ™sing elements. 
Similarly, an analyst or a researcher might infer that two 

if 1 lu.th"' al T ft r y , h T ,he same P erf °™nt ; e characteristics 

structural sLtaty ' ,n ‘ he the °f 

Classifying data also helps the researcher to uncover gaps in 

the "ST 3 ' 10 "- And discovering ^e existence of these Japs * 
the first step in generating future collection requirements 

In addition a classification system provides the researcher 
with a comprehensive overview of the structure of his data. It 

workin™ 2 Pattem ° r 3 Gestalt of the territory with which he is 

. . itisfor convenience that most researchers classify their 

cate wbh nf/ SyStem of c, ^ f ication permits one to communi¬ 
cate with others in generic or class names rather than having to 

refer to each specific entity within a class. For example, under 
most conditions “DD” or “destroyer” is considerably more 

Frc e h n eTciass” U rOf than Preston ’ N o. 795, Later 

rietcher Class (Of course, there are other times when the 

complete identity must be known.) 

Similarly, a classification system permits the researcher to 
free his mind of myriad details and to concentrate essentially on 
group characteristics or properties of the class of phenomena 

data fh h e Ch he h deaIlng ' Since classified data are also indexed 
with’. 1 C f y r neV6d eaSily ‘ Conse 9 uently, the researcher 
nn ,J a classiflca tion system has a device for filtering detail when 
only class properties are important, as well as a device for 
recalling specifics when needed. In short, classifying permits the 
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analyst to convert infinite amounts of data into finite, 
manageable elements that can be stored and retrieved. 

Another convenience a well-designed classification system 
provides is a method for accommodating new input without 
requiring extensive alteration of the files every time additional 
data are received. With an exhaustive classification system, the 
researcher can accommodate new data simply by expanding the 
categories he had already established. 

Finally, in terms of convenience to the user, classification 
systems permit any number of people to use the same system so 
long as the underlying rationale of the system is understood. 
The rationale of the system refers to the rules by which data are 
assigned to certain classes (or entered into the system), and to 
the rules by which data can be retrieved. In fact, once these 
rules are spelled out explicitly, the actual classification, storage, 
and retrieval of data can be automated. 

Coding: A Basic Type of 
Classification 

The most basic type of classification process, and perhaps 
the process most familiar to analysts and researchers, is the 
process of coding. Admittedly, coding is done primarily when 
large quantities of data are processed. But coding may also be 
performed profitably when relatively small projects are under¬ 
taken. Coding involves two basic steps: 1) organizing the data 
into classes, and, 2) assigning a number or a symbol to the item 
according to the class in which it falls. This relatively simple 
process is required whenever computers are used. Every datum 
entered into a computer’s data base is a classified and coded bit 
of information. But coding and classifying data helps the 
analyst in the manual analysis of data as well. 

The summary sheets in Figures XV-1 and XV-2 contain 
examples of personal data and responses to specific questions 
that might have been made during an interrogation of PW’s. 
Initially, information would be recorded on separate interview 
schedules or questionnaires. Obviously, data in this form are 
nearly impossible to handle. Furthermore, data on any single 
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Figure XV-1. Coded Responses From All Subjects 


form would reveal information from and about a single subject, 
but it would not reveal any pattern of responses from the group 
of PW’s as a whole. However, coding the responses on the basis 
of pre-determined classes permits the analysis of group re¬ 
sponses simply by tabulating categories of responses. 

Figure XV-1 is a record of all individual responses. Figure 
XV-2 shows how the same data can be aggregated for additional 
analysis. It can be seen for example that the responses from all 
company grade officers were consistent, but dissimilar to 
responses made by the other subjects; and that the responses 
made by infantry and artillery personnel tended to be similar, 
but unlike the responses made by support personnel. Admit¬ 
tedly, the sample size is relatively small, and the differences 
might not be significant statistically, but patterns of responses 
do emerge from tabulations of this nature. Should these data be 
entered into a computer, any number of runs could be made 
which might reveal additional patterns of responses that are not 
perceptible on the aggregated response tabulation; for example; 
correlations between age and rank, correlations between branch 
and age, or correlations between sector and branch of service. 

In the example just cited, the sample size (N) was only 89. 
If the sample size were considerably larger, data processing 
equipment should be used; for example: a key punch, sorter, 
and counter. 5 Of course, if any statistical analysis (other than 
counting) had to be performed on the data, a computer, or at 
least a hand calculator, would be needed. 

The CASCON Project: Coding and 

Classification Applied to 

Strategic Analysis 

A research project relating to local conflict that epitomizes 
the role that coding and classification play in analysis is the 
CASCON system (Computer-Assisted System for Handling 
Information on Local Conflict). 


5 The limit for convenient “hand processing” is about 150 subjects. 
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AGGREGATED RESPONSES OF PW INTERROGATIONS 


RESPONSES / 

PERSONAL DATA 


TEM 

1 

ITEM 2 

1 TEM 

3 

1 

TEM 

4 


1 TEM 

5 


A 

B 

C 

0 

A 

B 

C 

A 

B 

C 

A 

B 

C 

D 

A 

B 

C 

AGE (N) 



















15-19 (41) 

19 

10 


4 

9 

30 

2 

26 

1 1 

4 


12 

22 

7 

23 

15 

3 


20-24 (44) 

2 

22 


1 

34 


10 

23 

18 

3 

10 

27 

3 

4 

9 

33 

2 


25-29 (3) 


3 



3 



3 





3 



2 

i 


30 - (I) 


' 



1 



1 





1 



1 



TOTALS (09) 

40 

44 

0 

5 

47 

30 

12 

53 

29 

7 

10 

39 

29 

1 1 

32 

51 

6 


BRANCH 


















—j 

INFANTRY (71) 

30 

41 



39 

26 

6 

43 

27 


9 

36 

23 

3 

28 

41 

2 


ART 1LLERY(13) 

10 

3 



0 

4 

1 

10 

2 

1 

l 

3 

3 

6 

2 

9 

2 


COM SUPP (5) 




5 



5 



5 



3 

2 

2 

) 

2 


TOTALS 

40 

44 

0 

5 

47 

30 

12 

53 

29 

7 

10 

39 

2S 

1 1 

32 

51 

6 


SECTOR 



















1 (67) 

20 

39 



39 

27 

1 

42 

26 

1 

5 

25 

23 

4 

14 

47 

6 


II (15) 

12 

3 



6 

3 

5 

9 

3 

1 

3 

3 

3 

6 

11 

4 

0 


IN (7) 


2 


5 

1 


6 

1 


5 

2 

1 

1 

3 

1 

0 

0 


TOTALS 

40 

44 

0 

5 

47 

30 

12 

53 

29 

7 

10 

39 

29 

11 

32 

51 

6 


RANK 



















EM (70) 

39 

39 



46 

30 

2 

52 

25 

1 

1 

39 

20 

10 

32 

45 

1 


NCO (6) 

1 

4 


1 



6 


4 

2 

5 



1 


5 

1 


CO G (4) 




4 



4 



4 

4 






4 

/ 

FLD/G (1) 


) 



1 



1 





1 



1 



TOTALS 

40 

44 

0 

5 

47 

30 

12 

53 

29 

7 

10 

39 

29 

11 

32 

51 

6 

j 


Figure XV-2. Responses Tabulated by Category 


The system permits users to retrieve information on more 
than two dozen cases of local conflict stored in the data bank 
and to compare new cases of conflict with those already stored. 
The system was based on a model developed by Lincoln P. 
Blumfield at the Center for International Studies of MIT. The 
model distinguishes between various phases of conflict, 
various factors which would tend to escalate or to diminish the 
intensity of the conflict, and the measures associated with the 
factors. 

In order to analyze a new case of conflict on the basis of the 
cases in the data bank, the new case must be coded according to 
categories established for various phases. When the coding has 
been accomplished, data on stored cases can be retrieved for 
comparison. On the basis of comparisons with earlier and 
similar cases, specific policy measures for controlling the 
conflicts can be postulated.^ 

Two Meanings of "Classify" 

As a verb, classify has two meanings: in one sense, it refers 
to the act of assigning a specific datum to a predetermined 
position in a classification scheme. In another sense, classify 
relates to the act of breaking down data and organizing them 
according to related subclasses. Strictly speaking, this second 
operation is called division. 

To illustrate the first meaning of classify, i.e., to assign a 
specific datum to a predetermined slot, the activities of an 
intercept receiver operator who detects a radar signal can be 
cited. The operator might identify a signal as originating from 
an EW radar because the signal’s parameters were all associated 
with a particular group of EW radars. In this case, the 


6 For additional information on CASCON, see the Minutes of the 
Twelfth AMG Meeting: April 28, 1970, INR Analytical Methods Group, 
Department of State. 
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predetermined upper and lower limits of each signal parameter 
would constitute the criteria for group assignment.^ 

To illustrate the second meaning of classify, i.e., breaking 
down data and organizing them into classes, the activities of an 
analyst whose concern was “military vehicles” could be cited. 
Here the analyst could divide the general class of “military 
vehicles” into any number of subclasses. For example, he might 
classify vehicles on the bases of modes of propulsion, number of 
axles, general use, weight, and so on. The discussion that 
follows will relate essentially to this second meaning of the term 
(i.e., division ). 

The Steps in Classification 

The process of classification can be broken down into five 
steps. There are no hard and fast rules to follow, nor is there a 
strict sequence, although it is difficult to see how an object can 
be assigned to a class before the class criteria are established. 
The steps are as follows: 

1) The purpose or purposes the classification system is 
intended to serve must be determined. This is important 
because the purpose the system is to serve suggests the type of 
system that should be used. 

Students of taxonomy (the science of classification) recog¬ 
nize two basic types of classifications systems: natural systems 
and artificial systems. Natural classification systems are based 
upon the fundamental properties of the thing being classified, 
such as the number of parts, dimensions, color, or shape. 
Examples of natural systems would include the periodic table 
described previously, botanical classifications based on the 
floral parts of the plant, a classification of radars based on 
frequencies and other signal properties, a classification of 
aircraft on the mode of propulsion, number of engines, and so 
on. 


n 

Another name for a category would be a nominal scale. Classification 
in the first sense of the term defined above is actually a form of 
measurement, albeit a crude form, as discussed in Chapter XIV. 
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An artificial classification, on the other hand, reflects 
primarily the user’s needs. Examples of artificial systems would 
include a classification of radars on the basis of the country in 
which they were manufactured, a classification of aircraft as 
“bogey” or “friendlya classification of coasts and landing 
beaches according to the extent to which they permit landing 
operations, and so on. 

Halfway between the completely natural and the com¬ 
pletely artificial classification systems are the diagnostic 
systems. Diagnostic systems reflect both natural properties of 
the objects or phenomena being classified, as well as the needs 
of the classifier. Examples of diagnostic classifications would be 
classifications of roads and highways on the basis of their 
trafficability under varying climatic conditions and classifica¬ 
tions of radars based on the functions they perform. 

Index classifications are entirely artificial systems. With 
index classifications, accessibility of data is the prime considera¬ 
tion. In this sense, the user’s needs predominate in the selection 
of criteria for group assignment. Examples of index classifica¬ 
tions would include the Dewey Decimal System, the Library of 
Congress classification system, the index of any book, as well as 
any alphabetical arrangement of data. More relevant to intelli¬ 
gence would be classifications of ships by pendant numbers, 
alphabetical listings of heads of state, and the classification of 
G-2 journal entries based on times of message arrivals. 


To summarize: the purpose the classification scheme is 
intended to serve determines which type of classification system 
is most appropriate. 

2) When the purpose has been decided , the phenomena or 
data about the phenomena to be classified should be examined 
for those features or characteristics that could serve as possible 
criteria for classification. 

Here the researcher should ask himself what it is that the 
objects, events, or other phenomena have in common. What 
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features do they have that are distinctive or unique? The 
researcher should extend his imagination to permit considera¬ 
tion of all of the parameters of that which he is classifying. 
Some of the characteristics that come to mind may appear 
superficial initially. However, this step is essentially an explora¬ 
tory one for establishing all possibilities for classification. 

3) From the objects , events , or phenomena to be classified , 
those features which would satisfy the purpose the system is 
intended to serve should be identified . 

The function the classification is intended to serve should 
point to those features of the phenomena which are relevant for 
classification purposes. Invariably phenomena can be classified a 
number of different ways. The best way to classify is that way 
which satisfies a particular need of the user. For example, if a 
researcher is considering weapon systems with large kill radii, he 
might find that a classification based on the weapon system’s 
accuracy would be irrelevant since the kill radius would 
compensate for any lack of accuracy. 

The variety of ways by which missiles can be classified 
according to different needs is shown below. 

When range is an important consideration, missiles can be 
classified as long-range missiles (over 5000 nautical miles) or 
ICBM’s, intermediate range missiles (300-1500nm) or IRBM’s, 
or short-range missiles (under 300 nm). 

When speed is an important consideration, missiles can be 
classified as “subsonic,” “sonic,” “supersonic” (airspeeds 
ranging between Mach 1 and 5), and “hypersonic” (air speeds 
exceeding Mach 5). 

On the basis of control, missiles can be classified as either 
unguided or free-flight rockets, or guided missiles. Guided 
missiles, in turn, can be classified according to the nature of 
their guidance systems, for example, command guidance, dead 
reckoning, position-fixing, and seeker or homing guidance 
systems. 
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There are other ways to classify missiles, but the point 
should be clear: phenomena invariably can be classified a 
number of different ways depending upon the characteristics of 
the thing being classified, but most importantly, depending 
upon the needs of the classifier. 

4) Relationships among and between classes must be 
identified. 

This step involves two operations. It involves, first, deter¬ 
mining the qualitative criteria by which two things may be said 
to be related. This step then involves considering the higher and 
lower categories to which something can be assigned. Essentially 
this means that the researcher must determine (or establish) 
what entries would correspond to what rung on an imaginary 
ladder of abstraction. For instance,“Soviet Jet Fighter” is a 
class of airborne vehicles higher on an imaginary scale of 
abstraction than “MiG”, but lower on the same scale than 
“fixed wing aircraft.” 

In this respect scientists are fortunate. They have specific 
hierarchies which remain essentially invariant, such as the 
hierarchy of groups of living organisms, for example: 

KINGDOM 

PHYLUM 

CLASS 

ORDER 

FAMILY 

GENUS 

SPECIES 

But the intelligence analyst or researcher also has a number 
of ready-made hierarchical structures at his disposal. For 
example, he has the hierarchy of military units such as divisions, 
brigades, battalions, and so on; he has the hierarchy of military 
ranks; and he has the hierarchy of governmental structures. 
Creating an order of battle, for example, involves associating 
names and locations with hierarchical structures of military 
organizations. These frameworks serve as convenient pegs upon 
which names of people, places, and activities can be hung. 
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The analyst and researcher will encounter instances that do 
not require the consideration of hierarchies. For example, 
radars might be classified on the bases of frequency, PRF, 
pulsewidth, scan rate, and mode of polarization. In this case, 
those radars that had the same operating parameters would be 
grouped together. Radars operating on frequencies between 

1550 MHz and 3900 MHz, for instance, would be grouped 
together and identified as S-Band radars, or all of the kinds of 
radars associated with a given geographic area might be grouped 
together and then further classified according to function, e.g., 
GCI, EW, FC, SS, and so on. None of these classifications 
involves consideration of hierarchies. 

5) Classification criteria should be applied to the “popula¬ 
tion” and the entities should be assigned to specific groups. 
When this step has been accomplished, a cycle has been 
completed. This step is identical to the first sense in which the 
term “classification” is used, namely, to assign an entity to a 
class. 

Testing the Classification System 

Before a classification system is implemented on any large 
scale, it should be pilot tested. But even before testing, the 
system should be examined with respect to the following 
questions: 

1) Are the rules for assigning phenomena to categories used 
consistently? 

2) Are the categories or classes mutually exclusive i.e., do the 
categories overlap? (Phenomena may be classified many 
ways, but each mode requires a separate system.) 

3) Is the system complete for the purposes for which it was 
designed (i.e., are there elements to be accounted for 
which do not fit into existing categories)? 

4) Does every classification have at least two divisions? 
(Without two subdivisions, a class cannot be said to exist.) 
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Classifications as Abstracts of 

Reality with Parts Deleted 

In the preceding discussion of classification, emphasis was 
placed upon the “either—or” logic of a dichotomous scheme of 
classification. It should be remembered that all classifications 
schemes are arbitrary. Every classification system reflects the 
real world more or less. Although one can speak abstractly of an 
object or an event’s belonging to this category or to that, the 
real world seldom conforms to the niceties of a classification 
system. Often the classifier’s decision on group assignment is a 
toss-up. When the purpose of the system is kept in mind, little 
damage is done in arbitrary decisions. However, it is important 
to be consistent in order that the same phenomenon will always 
be classified the same way by all system users. 

Finally, it must be remembered that even for professional 
taxonomists or “systematizes”, the act of classification is not 
an end in itself. For intelligence research especially, it is only a 
beginning step in an analytic process. 

Summary 

E> Classification, or the process of assigning information to 
classes, is an essential step of nearly every type of research, 
many kinds of problem-solving activities, and all measurement. 

> Classification is performed for two main reasons: to gain a 
better knowledge of the data, and for convenience in storing 
and retrieving data. 

>The classification process typically involves five steps: 

- Establishing the purpose for which the classification will 
be performed 

- Examining the data about the phenomenon being classi¬ 
fied in order to establish bases for assigning a datum to a 
category 

- Selecting the bases for classification in light of the 
purpose for which the data were being classified 
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- Identifying relationships among and between classes of 
data, e.g., hierarchies, subclasses and the like 

-Assigning data to specific groups or categories 

[> Rules of classification require that 

- Phenomena (or data) be assigned to categories in a 
consistent manner 

- All classes or categories be mutually exclusive 

- The system must be complete in the sense that all 
elements of data can be assigned to a category and that 
there are no elements left over 

- Every superclass has at least two subclasses 


CHAPTER XVI 

BASIC QUANTITATIVE TECHNIQUES FOR 
RESEARCH AND ANALYSIS 


When you can measure what you are speaking about, 
and express it in numbers, you know something about it; 
but when you cannot measure it, when you cannot 
express it in numbers, your knowledge is of a meagre 
and unsatisfactory kind; it may be the beginning of 
knowledge, but you have scarcely advanced to the stage 
of science. 

Lord Kelvin 

At some point in his career, practically every researcher or 
analyst will work with quantitative data. In some instances, the 
researcher personally may have to perform certain types of 
quantitative analyses. In more instances, the researcher will have 
to utilize the results of quantitative analyses performed by 
other researchers. This chapter reviews basic statistical and 
quantitative techniques that are relevant to intelligence research 
and analysis. In some cases only a principle is addressed, and the 
application of that principle is left to the reader. In other cases, 
particularly where a clearly identifiable technique is discussed, 
the application of that technique to an intelligence problem is 
cited. 

To the researcher unfamiliar with mathematical symbology, 
some of the equations used in this chapter might appear arcane 
and formidable. However, the operations required are usually 
quite simple and anyone who has had some basic algebra should 
have no problems with the equations. The purpose of the 
chapter is to acquaint the researcher with basic fundamentals of 
statistics. It is not the intent of this chapter to train the reader 
to develop or apply a statistical technique to a specific problem. 
Entire texts are devoted to that objective. 
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Statistical Analysis 

Statistics is the science of gathering, analyzing, and drawing 
inferences from masses of numerical data. There are three main 
types of statistics: descriptive statistics , sampling theory , and 
inferential statistics. Descriptive statistics indicate the magni¬ 
tude and spread of a set of data. In the case of a suspected 
mobilization, for example, descriptive statistics could tell the 
researcher what the average number of flatcars carrying heavy 
equipment by train from, say, Volkovysk to Bialystok had been 
over the last several months. (The researcher could also 
compute a separate average number of flatcars returning 
empty.) 

Sampling theory allows one to make inferences about many 
(or all) instances, occurrences, events, or conditions (the 
“population”) on the basis of only a part of that population. 
The researcher who is concerned with the movement of materiel 
from the Soviet Union to Poland, for example, might know that 
four trains travel over a selected route daily in each direction 
(the entire population), but he might count the flatcars on the 
train only twice each week (his sample). Sampling theory 
permits him to plan the pattern of his observations and to 
decide how large his sample should be. 

Inferential statistics perform several functions which con¬ 
cern drawing conclusions about a set or sets of data. Some 
inferential statistics allow a researcher to express quantitatively 
the degree of confidence he can place in his generalization of an 
entire population based on sampled data of that population. 
For example, the researcher might discover that the proportion 
of trains moving westward with loaded flatcars increased each 
successive week, and conversely, so did the number of trains 
moving eastward with empty flatcars also increase. Inferential 
statistics could indicate if the same trend would be evident 
whether observations were taken each day or once each week. 

Statistical inference can also examine the relationships 
between two or more variables. If the number of loaded flatcars 
traveling west were no longer simply noted but were graphed 
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according to time (Figure XVI-1), then time as well as the 
number of flatcars would become an important variable. Using 
linear regression, a line could be drawn to indicate the trend 
toward increasing numbers of flatcars over time. Statistical 
inference could also be used in the same example to explore the 
relationships among other variables; for example, the overall 
lengths of the trains, and the number of passenger cars on each 
train. 

Mathematical Analysis 

Although statistical analysis is especially useful to intelli¬ 
gence research and analysis because conclusions must often be 
drawn from partial data, there are times when it is better to use 
simple mathematical formulas. For example, if a commander 
determined that he had engaged elements of two separate 
battalions, he could estimate the size of the forces opposing 
him simply by multiplying the strength of each unit times the 
number of units engaged. If he knew that the two battalions 
with which he was engaged were part of a regiment, he could 
also estimate the size of the overall force with which he might 
be engaged in the near future by multiplying the total number 
of units in the regiment by the strength of each unit. In this 
case, simple arithmetic would suffice for making an estimate. 

Descriptive Statistics 

Descriptive statistics are measures that are used to sum¬ 
marize large amounts of numerical information and to create 
profiles describing the information. Descriptive statistics can be 
expressed graphically, verbally, and mathematically. In the 
following section, methods and measures of descriptive statistics 
are introduced. They include frequency distributions , measures 
of central tendency , and measures of dispersion. 

Frequency distributions 

Specific measures of traits or characteristics are called 
values. A trait or characteristic that assumes different values is 
called a variable. For example, a researcher might be interested 
in the heights, ages, wealth, educational levels, or years of 
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Figure XVI-1. Data Noted and Data Graphed According to Time 


214 


military service of a group of people. The values of these traits 
vary among the individuals in the group; hence, they are called 
variables. 


A tally might be made of the numbers of people of each 
category of height, age, or any other variable. For example, the 
heights of 292 American soldiers could be measured, and the 
tally might look like this: 


Height 

Number 

(inches) 

of that 

60-62 

12 

62-64 

28 

64-66 

37 

66-68 

42 

68-70 

50 

70-72 

44 

72-74 

36 

74-76 

25 

76-78 

15 

78-80 

6 


The heights of the soldiers are recorded according to 2-inch 
intervals; thus, a soldier who is 68 inches tall cannot be 
distinguished from a soldier who is 69 inches tall by looking at 
the tally sheet. It is important for the researcher to choose the 
correct size of a class interval. The size of the interval is 
determined by the uses to which the data will be put. In this 
example, a one-eighth inch class interval would be a poor choice 
because an individual’s height often varies more than one-eighth 
of an inch during a single day. On the other hand, a 20-inch 
class interval would not show the researcher any pattern at all. 
It is also advisable^ for most purposes, to have all class intervals 
the same size (e.g., all 2-inch intervals rather than some 2-inch 
and some 4-inch intervals). 

The tally sheet indicates that many of the soldiers are 68 
inches to 70 inches tall, whereas few are either very tall or very 
short. But the tally sheet does not give the reader a concise 
visual image of the pattern of heights. Graphing the data with 
heights recorded on an horizontal axis and the numbers of 
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individuals recorded on a vertical axis (Figure XVI-2), however, 
does produce a visual image of the data. 

A graph, such as Figure XVI-2, which is a series of 
rectangles, is called a histogram. The image may be enhanced by 
joining the midpoints of the tops of each rectangle. The 
histogram shows the number of individuals constituting each 
class, the relative size of each class, and the distribution of the 
classes. The roughly symmetrical “curve” of the histogram 
indicates that there are about as many short soldiers as there are 
tall ones. 

Measures of central tendency 

Weapons are usually made in only one size. Weapons must 
be designed so that it is possible for the tallest and the shortest 
soldier to use them. Therefore, a weapon is usually designed to 
suit the “typical” or “average” soldier. 1 

The “typical” soldier’s height is computed as one of three 
measures of central tendency: the mean , median , or mode. The 
mean is an average which may be computed by adding up all the 
individual values of a variable and then dividing that sum by the 
number of values. In the example of soldiers’ heights cited 
earlier, there were many soldiers of the same height. Rather 
than adding each height individually, it is more efficient to 
calculate the mean in the manner shown below (Table XVI-1). 

The middle height of each class interval, e.g., 61 inches as 
the middle of the class 60 - 62 inches, is multiplied by the 
number of soldiers of that height to produce the third column 
or the total number of “soldier-inches” in that class interval. To 
find the mean, columns (2) and (3) are each summed, and then 
the total of column (3) is divided by the total of column (2): 


lr This is no trivial matter. One reason for the popularity of the M-l 
carbine in Asia was its small size and light weight. 
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Figure XVI-2. Height: American Soldiers (Inches) 


















































Table XVI-1. Soldiers’ Heights and Total “Soldiers-Inches” 


(1) 

HEIGHT 
(INCHES) 

(2) 

NUMBER OF SOLDIERS OF 
THAT HEIGHT 

(3) 

TOTAL SOLDIER-INCHES OF 
THAT HEIGHT 

60 - 62 

12 

732 

62 - 64 

28 

1764 

64 - 66 

37 

2405 

66 - 68 

42 

2814 

68 - 70 

50 

3450 

70 - 72 

44 

3124 

72 - 74 

36 

2628 

74 - 76 

25 

1875 

76 - 78 

15 

1155 

78 - 80 

6 

474 


295 

20421 
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Total soldier-inches _ 20421 = 59 2 inches 
Total soldiers 295 

The median refers to the middle value of a variable. For 
example, if all of the soldiers were lined up in order of 
increasing height, the height of the soldier who was in the 
middle of the line would be the median. The median is found 
by arraying all the data again (Table XVI-2). 

In column (3), the number of soldiers in each class is 
accumulated with the number of soldiers in all previous classes 
(e.g., 12 + 28 + 37 . . . ). Since there are 295 soldiers, the 
soldier of median height would be the 148th soldier. Column 
(3) indicates that the 148th soldier would be somewhere 
between 68 and 70 inches tall. 

The mode is the class which occurs most frequently. In the 
current example, the mode is in the class 68 - 70 inches. (Fifty 
soldiers fall in this class.) 

If the United States were preparing to send weaponry or 
other military equipment to an ally, a comparative statistical 
analysis of the heights of American servicemen and their allied 
counterparts could aid planners in determining what sizes of 
uniforms or what types of weapons to provide. 

For example, a histogram of the heights of 225 allied 
soldiers (Figure XVI-3) indicates that they are much shorter, in 
general, than their American counterparts. The mean of the 
distribution is 62.4 inches, whereas the median and mode both 
fall between 62 and 64 inches. 2 Therefore, to fit the typical 
allied serviceman, uniforms would have to be issued that were 
designed for a man 7 inches shorter than the typical American 
soldier. 


2 In normally distributed data, the mean, median , and mode are 
roughly equivalent. 
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Table XVI-2. Soldiers’ Height Data Rank-Ordered 


(1) 

HEIGHT 

(INCHES) 

(2) 

NUMBER OF SOLDIERS OF 
THAT HEIGHT 

(3) 

CUMULATIVE 

NUMBER OF SOLDIERS 

60 - 62 

12 

12 

62 - 64 

28 

40 

64 - 66 

37 

77 

66 - 68 

42 

119 

68 - 70 

50 

169 

70 - 72 

44 

213 

'72-74 

36 

249 

74 - 76 

25 

274 

76 - 78 

15 

289 

78 - 80 

6 

295 


295 
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Figure XVI-3. Heights of Allied Soldiers (Inches) 










































Measures of dispersion 

The heights of the group of American soldiers are spread 
over 20 inches (60 - 80 inches). If a different sized uniform 
were needed for the soldiers of every 2-inch class of height, then 
10 different uniform sizes would be needed for the entire 
group. The difference in height between the tallest and the 
shortest allied soldier in the group (the range of heights) is only 
18 inches. Using the same criterion as above, only 9 uniform 
sizes would be needed for the entire group. 

Data which are spread widely about an average value are 
considered to be “dispersed.” Histograms which are relatively 
tall and narrow show less dispersion than shorter, wider 
histograms. One of the more commonly used measures of 
dispersion is called the standard deviation. The standard 
deviation can express how close most of the soldiers are to the 
mean height of the group. 

The formula for the standard deviation is: 


o = 



(Xj - x) 2 
"ll 


The formula involves the following steps: 1) the mean (x) 
must be subtracted from each individual value (xj); 2) the 
results of each (xj - x) must be squared; 3) all the (xj - x) 2 are 
summed; 4) the result is divided by the total number of items; 
and 5) the square root of the result is taken. 

The calculation of the standard deviation for the allied 
soldiers’ heights is shown on the following page. 
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(1) 

MIDPOINT OF 
HEIGHT CLASS 

<v 

(2) 

(X. -7) 

x= 62.4 

(3) 

(X. - x) 2 

(4) 

NUMBER OF 
SOLDIERS OF 
THAT HEIGHT 

(5) 

(3) X (4) 

55 

-7.4 

54.76 

12 

657.12 

57 

-5.4 

29.16 

25 

729.00 

59 

-3.4 

11.56 

24 

277.44 

61 

-1.4 

1.96 

40 

78.40 

63 

.6 

.36 

45 

o 

CSI 

CO 

65 

2.6 

6.76 

35 

236.60 

67 

4.6 

21.16 

25 

529.00 

69 

6.6 

CO 

to 

CO 

15 

653.40 

71 

8.6 

73.96 

4 

295.84 


= 225 

= 3473.00 

n 

= 3.93 
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The midpoint of each class interval is used to represent the 
interval, as it was when the mean was calculated. Since there are 
many soldiers who are the same height, the (xj - x) 2 are 
multiplied by the number of soldiers of a given height, and 
those results are summed (Column 5). This is a short cut similar 
to the one used in the calculation of the mean. 

The standard deviation in height of the allied soldiers is 3.93 
inches. This measure indicates that most of the soldiers are 
taller than 58.5 inches but shorter than 66.3 inches. If the 
distribution of heights formed a normal curve (Figure XVI-5, 
page 231), then 68 percent of the heights would be included 
within 1 standard deviation of the mean and 95 percent of the 
heights would fall within ±1.96 standard deviations of the mean. 
In this example, the distribution of heights forms a curve similar 
enough to a normal, bell-shaped curve so that it can be assumed 
that roughly two-thirds of the heights fall within 1 standard 
deviation of the mean. 

The standard deviation of the distribution of American 
soldiers’ heights is 4.07 inches. Therefore, most of the American 
soldiers are taller than 65.1 inches but shorter than 73.3 inches. 
In order to compare the dispersion of the two distributions, the 
standard deviation should be divided by the means. This 
standardizes the two measures of dispersion. The division is 
necessary because a difference of 1 inch from the mean for the 
shorter allied soldier is a greater percentage difference from the 
mean than is the case of the American soldier who is 1 inch 
taller than the mean. 

The relative measure of dispersion is called the coefficient 
of variation and is equal to 


a_ = 3.93 
x 62.4 


.063 


for the allied group, and equals 
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a 

x 


4.07 

69.2 


= .059 


for the American group. 

This measure shows that the “allied” distribution is more 
dispersed than the “American” distribution. It should be noted 
that although the range and the standard deviation of the 
American soldiers’ heights were greater, the coefficient of 
variation of the “allied” heights data was greater. The range and 
coefficient of variation measure different aspects of dispersion. 

An Example of the Use of 

Descriptive Statistics 

The United States and the Soviet Union constantly monitor 
the comparative strengths of their defense establishments. It is 
difficult to compare fleet strengths in the two nations’ navies 
because the classes of ships are defined differently. As a 
preliminary part of the analysis of the two fleets, the number 
and size (displacement) of frigates in the two navies might be 
compared (Figure XVI-4). The histogram representing the ships 
indicates that the definition of frigates differs greatly. The 
Soviet frigates are much smaller than the American frigates, and 
there are also many more Soviet frigates. 

A “typical” frigate for each navy is described by utilizing 
the figures below (Table XVI-3). 

The mean size of the U.S. frigates is 6,379 tons, the median 
size is 5,670 tons, and the mode is 4,700 tons. In this 
distribution, which does not conform to a bell-shaped curve, 
values of the mean, median, and mode are quite different. The 
mean size of the Soviet ships is 1070 tons, and the median and 
mode are both 950 tons. The coefficients of variation are .2723 
for the U.S. ships and .1463 for the Soviet ships. Therefore, the 
class of U.S. ships called frigates is a relatively broader class 
than Soviet frigates. 

The American ships known as ocean escorts are more 
comparable to Soviet frigates than are American “frigates.” A 
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Figure XVI-4. Comparison of Frigates, U.S.S.R. Navy, U.S. Navy 


Table XVI-3. Frigates of the U.S. and USSR Navies 


FRIGATES OF THE UNITED STATES NAVY 

(D 

DISPLACEMENT 

(TONS) 

(2) 

NUMBER OF SHIPS 

(3) 

(D X (2) 

10,150 

2 

20300 

10,000 

3 

30000 

8,200 

1 

8200 

7,600 

1 

7600 

6,570 

9 

59130 

5,670 

9 

51030 

4,700 

10 

47000 


35 

223260 

FR 

IGATES OF THE U 0 S,S.R. 

NAVY 

(D 

DISPLACEMENT 

(TONS) 

(2) 

NUMBER OF SHIPS 

(3) 

(1) X (2) 

1500 

6 

9000 

1200 

40 

48000 

950 

20 

19000 

950 

45 

42750 


111 

118750 
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histogram of ocean escorts would show that there are some 
ocean escorts about the same size as Soviet frigates although the 
U.S. class of escort ships is generally somewhat larger. The 
mean, median, and mode sizes of American escort ships are 
3494, 4100, and 4100 tons, respectively. 

Similar analyses might be undertaken for all the types of 
ships and various characteristics of those ships in both navies. 
The resultant profile of fleet strengths would then be a concise 
method of comparing the fleets on the basis of displacement 
and class. 

Sampling Theory 

There are many times when an analyst or a researcher would 
want to develop a profile of the “other side’s” activities. 
Profiles are essential for detecting deviations from the norm. 
Without profiles (or other baseline data), it is impossible to 
determine whether a certain event or condition is typical or 
atypical. 

For example, an intelligence analyst might want to develop 
a profile which described the types of information transmitted 
by the wire services of a specific country. He would want the 
profile to be representative of a “typical” day, so he would try 
to review wire service releases transmitted over a period of time. 
If the analyst could not read every story that was transmitted 
by the wire services, he would choose only a portion of the 
stories for careful review. 

All of the items of interest to the researcher might include 
every release transmitted by the wire services during a specific 
month. This entire body of data is known as the population or 
universe. The part of the entire population which is selected for 
analysis is called a sample. If a sample is selected carefully, it is 
possible to make inferences about characteristics of the overall 
population based upon characteristics of the sample. There are 
several methods for selecting samples, and the method that is 
chosen depends upon the purpose for which the sampling is 
being conducted. 
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Random sampling 

A random sample gives each member of the population an 
equal chance of being chosen. Often a random number table is 
utilized for selecting a random sample. Other procedures 
include drawing numbered slips of paper out of a hat or tossing 
dice. 

In the current example, a random number table could be 
used to help choose when wire service releases would be 
reviewed at length. It might be predetermined that ten releases 
would be reviewed on a given day. The ten releases would be 
chosen by consulting the random number table and noting the 
last two digits of ten five-digit blocks of numbers. Reading 
across the first row, the ten two-digit numbers might be 56, 57, 
42, 22, 06, 45, 55, 05, 30, and 80. For the given day, then, the 
releases to be reviewed would be the fifth, sixth. . . . eightieth 
items transmitted after 2400 hours. 

An even simpler sampling scheme for this example might 
utilize the time/date group in the header data of each article; 
for example, the release transmitted each hour on or about 37 
minutes after the hour might be selected. Although acceptable, 
this type of sampling sometimes introduces bias. For example, 
if radio broadcasts were monitored, and if the analyst examined 
the transmissions at 55 minutes after every hour, he might 
discover that sports or weather were always reported at that 
time. A profile based on this biased sample of radio broadcasts 
would suggest that sporting events and weather information 
were reported 100 percent of the time that the station was on 
the air. 

A major advantage of random sampling is that it allows one 
to assess the accuracy of the measures of central tendency and 
measures of dispersion derived from sampling. Thus, an analyst 
is able to answer the question: How confident am I that the 
mean I calculated from sampling is within a certain range of the 
mean of the population? 

The example of the heights of allied soldiers may serve to 
show the usefulness of establishing confidence levels. The 225 
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allied soldiers whose heights were measured were a random 
sample taken from a population of, perhaps, millions of 
soldiers. The mean of their heights was 62.42 inches. This mean 
height may be the same as the mean height of the population. 
On the other hand, the analyst may be challenged by someone 
who stated that the average allied soldier was 64 inches tall. The 
analyst could establish a confidence level for his calculation of 
the mean height in the following way: 

He knows his sample size, sample mean, and sample 
standard deviation: 


n = 225 
x = 62.42" 
o = 3.93" 

From the mathematical theory associated with random 
sampling, he also knows that there is a chance that his sample 
mean may be either higher or lower than the true mean for the 
population. If several samples are taken from a population, 
most sample means will be too large or small by a small amount, 
but some samples will be a great deal too large or too small 
(Figure XVI-5). 

In Figure XVI-5, if x pop is the true mean of the 

population, many sample means will be nearly equal to the true 
mean (there are 33 samples with a mean of x 2 , just slightly 
larger than the true mean). Only a few sample means will be 

very inaccurate (there are only 8 samples with a mean of xj, 

much smaller than the true mean). The distribution of errors of 
sample means around the true mean forms a normal or 

bell-shaped curve (Figure XVI-5). 

The standard error (o%) of the sample mean must be 
computed to determine how likely it is that the estimated 
average height of 62.42 inches falls within a certain range of the 
true mean height. The formula for the standard error is: 
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Curve) 


If the o of the sample is used as the estimate of the standard 
deviation of the population, this formula becomes: 


3.93" = 3.93" 
225 15 


.262" 


To establish confidence levels for the estimated average 
height of 62.42 inches, a table of areas under the normal 
distribution must be consulted. The table tells the analyst how 
likely it is that his estimated mean is within a range of the true 
mean. For example, if the analyst specifies that the range of 
heights is to be 
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to 


62.42"-1 o- or 62.42"-.262" = 62.158" 


62.42" +1 a* or 62.42" + .262 = 62.682", 

the table tells him that the likelihood that the true mean falls 
within this range is 68 percent. 

Alternatively, an analyst can specify that he wants to find 
the range of heights so that there is a 95 percent likelihood that 
the true mean falls within an interval. The analyst would 
consult the table and find that the 95 percent confidence level 
would be 1.96 standard deviations above and below the sample 
mean. The boundaries of the range are then set: 

62.42" - 1.96 a x or 62.42" - .514" = 61.906" 

62.42" + 1.96 cr_ or 62.42" + .514" = 62.934" 
x 

As the confidence level increases , in this case from 68 percent 
to 95 percent, the size of the range also increases. 

Random sampling schemes also allow the researcher to 
determine how many data points (value of n) must be included 
in the sample in order to be certain of a specific level of 
confidence in his results. 

Stratified sampling 

The object of stratified sampling is to choose a cross section 
of the population. Stratified sampling is accomplished in two 
steps. Initially, the population would be classified into sub¬ 
groups which tended to share similar values for the character¬ 
istics under study, and the size of each subgroup would be 
recorded. A random sample would then be taken within each 
subgroup, with the sample size determined by that subgroup’s 
percentage of the total population. 

For example, it might be necessary to assess the opinions of 
villagers toward a military civic action program—the construc¬ 
tion of a bridge perhaps. If it were not possible (nor desirable) 
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to interview every individual in the village, the number and type 
of subjects to be interviewed could be established by first 
classifying groups of villagers according to age, sex, and 
occupation (in this example), and then determining how many 
subjects constituted each group. The percentage of the total 
village population that each group constituted would be 
calculated. Then, for each 5 percent of the population 
contained in each group, one subject would be selected to be 
interviewed. Thus, in the example below, children constituted 
30 percent of the total village population. Therefore, 6 children 
would be selected randomly as being representative of all of the 
children in the village. In this example, choosing 1 subject for 
each 5 percent of the total population would yield a sample size 
of about 20. (Controlling the sample size helps the analyst to 
estimate his total interviewing time.) 


Table XVI-4. Stratified Sampling Scheme, Village Population 


GROUPS OF INDIVIDUALS IN 
THE VILLAGE CLASSIFIED BY 
SEX,AGE,AND OCCUPATION 

NUMBER OF 

OF PEOPLE 

PERCENT OF 
POPULATION 

(*) 

NUMBER IN 
SAMPLE 

CHILDREN (UNDER AGE 15) 

64 

30 

6 

WOMEN (AGES 15 - 50) 

60 

28 

6 

MEN (AGES 15 - 50) 

50 

24 

5 

-FARMERS 

(33) 

(16) 

(3) 

-SMALL BUSINESSMEN 

(11) 

(5) 

(D 

-OTHER 

(6) 

(3) 

0) 

ELDERLY (50 OR OLDER) 

39 

18 

4 

TOTAL 

213 

100« 

21 
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In order for at least 1 person to be interviewed from each 
group, at least 1 man in the category “other” would have to be 
interviewed. Since the category “other” represents only 3 
percent of the population, the category is slightly overrepre¬ 
sented in the sample. 

If the division of the population into subgroups (classifica¬ 
tion) is done wisely, this type of stratified sampling will yield a 
better cross section of the population than most random 
samples. However, confidence levels cannot be assessed for the 
values yielded from stratified samples. Also, much must be 
known about the population in order to define the subgroups, 
and in instances where demographic data are lacking, this may 
be a problem. 

Introduction to Probability 

Probability theory provides the mathematical foundation 
for inferential statistics. Probability theory generates 
“idealized” distributions of the frequency of occurrence of 
chance events. 

The normal curve is one example of such a distribution 
(Figure XVI-6). In an “ideal” case, the relative frequency of 
specific values of Xj could be plotted accurately so that the six 
selected values of xj lie on the normal curve (Figure XVI-6A). 
Real-world data rarely fit ideal mathematical models exactly. 
Events occur too frequently or not frequently enough with 
respect to the ideal model. For example, in Figures XVI-6B and 
6C, the six selected values lie above or below the curve. 
However, the two sets of data represented by Figures 6B and 6C 
approximate the normal curve closely enough so that the curve 
can serve as a basis for analyzing the data. 

Probabilities range from impossibility (a probability of 0) to 
absolute certainty (a probability of 1). With probability theory, 
an event must be considered as occurring or not occurring. 
Given that p is the probability of the occurrence of event E, the 
probability that E will not occur is equal to 

q = 1 — p (1) 
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Figure XVI-6. A-The Normal Distribution. B,C-Approximations 
of the Normal Distribution 














For example, the Democrats, Republicans, Liberals, and 
Conservatives each have a candidate running for the U.S. 
Senate. Assuming that the two former candidates each have 
about twice as great a change of winning the election as each of 
the two latter candidates, and assuming that the election is 
certain to be held, then 

Pr[D] + Pr[R] +Pr[L] +Pr[C] - 1, or 1 + 1 + 1 + 1 = 1. (2) 

6 6 6 6 

The probability of the Liberal candidate’s not winning is 


PrfnotL ] =1+2+1 = 5 ( 3 ) 

6 6 6 6 

Equation (3) is an example of the additive law of 
probability , which states that the probability of any one of 
several mutually exclusive events occurring equals the sum of 
the separate probabilities of the events. 

Another law of probability, the multiplicative law, states 
that the probability of all of several independent events’ 
occurring in succession equals the product of the separate 
probabilities of the events. In the election example, the 
multiplicative law could be applied to the probability of two 
Liberal candidates’ winning the Senatorial race in two states. 

Assuming theprobability that the Liberal will win each state isl; 

6 

Pr[L in both] lxi= I (4) 

6 6 36 


Probability Distributions 

Data which can be counted, like the number of people in a 
city, are called discrete data. Data which can occur anywhere 
within a range, like the speed of a car, are called continuous 
data. Intelligence researchers must sometimes measure both 
discrete and continuous phenomena. There are several 
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important probability distributions for each type of phenom- 
ona, but only three will be discussed at length: the normal 
distribution, the binomial distribution, and the exponential 
distribution. 

All probability distributions can be illustrated graphically 
by plotting the number of events on one axis and the 
probability of occurrence of these events on the other axis. The 
relationship between a histogram showing the frequency of 
certain data values and the probability distribution associated 
with the same set of data is explained below using the example 
of intelligence (I.Q.) test results. 

A histogram showing intelligence test results (Figure XVI-7) 
illustrates the distribution of intelligence quotients for a given 
sample group. It shows that there is a large group of people with 
average I.Q.’s. The histogram is also nearly symmetrical; there 
are nearly equal numbers of people with high I.Q.’s and with 
low I.Q.’s. 

Normal distribution 

The appearance of the histogram tells the experienced 
researcher that the distribution can be approximated by a 
normal distribution. When the normal distribution is used as the 
model for the data, further analysis is based upon the normal 
distribution rather than upon the original histogram of data. 
The fitting of real-world data to ideal probability distributions 
is often associated with taking a sample from a large population. 
For example, the normal distribution (Figure XVI-7) is used to 
describe intelligence test results from a sample of 230 people 
out of a population of 10,000 people. The researcher is 
assuming that the distribution of I.Q.’s in the entire population 
is normal. The slightly irregular distribution of I.Q.’s in the 
sample arises only because the sample is not quite representative 
of the population. 

The normal distribution is a continuous distribution. It 
describes many types of data well, including abilities, heights of 
people, and the slight irregularities in the sizes of a product 
produced by machine. 


237 










238 


Figure XVI-7. Distribution of I.Q.’s 


Binomial distribution 


The binomial distribution describes discrete data. It arises 
when events depend upon a fixed probability of occurrence, p, 
and when the number of trials, n, is limited. For example, 
planners might be concerned with resupplying insurgents in a 
hostile area by an airdrop. They might know that the insurgents 
constitute only one-third of the total population (p = 1). They 
would be interested in the likelihood of reaching at least 5 
insurgent groups if 10 bundles of supplies were dropped. 

Figure XVI-8 shows the binomial distribution when n = 10 

an d p = i- 
3 ‘ 

In order to find the probability of reaching at least 5 groups 
of the target population when 10 bundles are dropped, the 
additive law of probability must be applied to the binomial 
distribution (see page 236): 

Pr[at least 5] = 

Pr [ 5 ] + Pr[6] + Pr[7] + Pr[8] + Pr[9] + Pr[ 10] (1) 


Pr[at least 5] = .136 + .057 + .016 + .003 + .0003 + 

.00002 = .21232 (2) 

The probability of reaching at least 5 groups of the target 
population is only .21 or about 1 chance out of 5. Such a 
simple analysis does not take into account the size of either the 
total population or the target population. The population size 
would become important if, for instance, the objective of the 
mission were to reach at least 50 percent of the target 
population. 

Exponential distribution 

The exponential curve provides an accurate description of 
the likelihood of occurrence of an unlimited number of events. 
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Figure XVI-8. An Example of a Binomial Distribution 


For example, the length of time between the arrivals of 
aircraft on a carrier may be called an event. If 2 aircraft arrive 
simultaneously, the length of time between their arrivals is zero 
(0). Therefore, 


Event] (time between arrivals) = 0. 

If the next aircraft arrives 2 minutes later, then 

Event2 (time between arrivals) = 2. 

In certain situations--for example, after an airstrike--it 
would be common for several aircraft to arrive at the carrier 
almost simultaneously. Therefore, the most common value for 
the event (time between arrivals) is zero (0). So, in a probability 
distribution describing aircraft arrivals, the value zero (0) has 
the highest probability. It is less probable for there to be 1 
minute between arrivals and even less probable for there to be 5 
minutes between arrivals. 

The duration of a message transmission might also be called 
an event. It is most likely that a message will be of a very short 
duration, less likely that a message will last 3 minutes, and even 
less likely that a message will last 6 minutes. The duration of 
message transmissions is very well described by the exponential 
distribution. 

Another common use of the exponential distribution is the 
description of the lifespan of pieces of equipment. For example 
(Figure XVI-9), all radio transmitters in a lot are checked before 
leaving the factory and all work properly. One year later most 
of them still work, but at the end of 4 years very few work 
(without having been repaired). 

The choice of a particular distribution depends upon the 
nature of the phenomenon to be described. A duration of time, 
for example, is often described by the exponential distribution, 
whereas the apportionment of abilities is described by the 
normal curve. Probability distributions are powerful tools for 
prediction because they tell a researcher which values of an event 
are most likely to occur. 
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The intent of these brief descriptions is not to teach how to 
create or develop the distributions, but rather to describe types 
of distributions that might be highly relevant to a problem. For 
information on how to develop and apply these distributions, 
the reader should consult the references at the end of the 
chapter. 

For more information about basic statistics, Lester Guest’s 
Beginning Statistics 3 , or L. H. Longley-Cook’s Statistical 


3 Lester Guest, Beginning Statistics (New York: Thomas Y. Crowell 
Company, 1957). 
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Problems and How to Solve Them 4 would be useful. For the 
more advanced student of statistics, J. P. Guilford’s Psycho- 
tncfric Methods 5 , and Robert V. Hogg and Allen T. Craig’s 
Introduction to Mathematical Statistics ° would be appropriate. 
For the researcher who is also concerned with experimental 
design, B. J. Winer’s Statistical Principles in Experimental 
Design 7 should be consulted, as should E. F. Lindquist’s Design 
and Analysis of Experiments in Psychology and Education a 
classic in its field. Finally, for the novice and professional 
researcher alike, Darrell Huffs short and slightly irreverent 
book, How to Lie With Statistics 9 , is most valuable for 
illustrating the uses and abuses of statistics for rhetorical 
purposes. 

Summary 

t> Statistics is the science of collecting, analyzing, and drawing 
inferences from masses of numerical data. 

t> Quantitative data do not always require statistical analysis. 
At times a simple mathematical formula suffices for an analysis 
of quantitative data. 


^L. H. Longley-Cook, Statistical Problems and How to Solve Them 
(New York: Barnes & Noble, Inc., 1970). 

5j. P. Guilford, Psychometric Methods (New York: McGraw-Hill 
Book Company, Inc., 1954). 

^Robert V. Hogg and Allen T. Craig, Introduction to Mathematical 
Statistics (New York: The Macmillan Company, 1970). 

^B. J. Winer, Statistical Principles in Experimental Design , 2nd ed. 
(New York: McGraw-Hill Book Company, Inc., 1971). 

^E. F. Lindquist, Design and Analysis of Experiments in Psychology 
and Education (Boston: Houghton Mifflin Company, 1953). 

^Darrell Huff and Irving Geis, How to Lie With Statistics (New York: 
W. W. Norton & Co., Inc., 1954). 
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t> Descriptive statistics permit a researcher to create an image 
of the data distribution. Descriptive statistics include: 

Frequency distributions (or profiles of data values) 

- Histograms (frequency distributions in the form of bar 
graphs) 

Measures of central tendency: mean, mode, and median 
Measures of dispersion or spread of data: standard 
deviation and coefficient of variation, and others 

t> Sampling theory allows one to generalize about population 
characteristics on the basis of an analysis of only a small part of 
the population. 

Two common sampling types are random sampling and 
stratified sampling. 

An advantage of random sampling is that it permits one 
to ascertain how likely it is that the sample has similar 
characteristics to the whole population. 

I> Probability theory provides the mathematical foundation 
for inferential statistics. 

Probability theory makes it possible to generate “ideal” 
distributions of data for purposes of comparison with 
actual distributions of data 

Three common distributions are illustrated by the 
normal, binomial, and exponential curves 
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CHAPTER XVII 

DESCRIPTIVE ANALYSIS METHODOLOGIES 


As our graph shows, Trends are trending. This is good. 

Yet, nevertheless, the destination of the trendings is not 
simple to assess. 

Dr. Seuss 

Descriptive and predictive research constitute the major 
types of intelligence analysis and research activities, as pointed 
out earlier. Just as the distinction between description and 
prediction is somewhat tenuous, so are distinctions between 
descriptive and predictive methodologies tenuous. 

It will be recalled that, strictly speaking, the term “predic¬ 
tive research” should be limited to those activities relating to 
developing and validating conceptual models which could be 
applied to a variety of future-oriented problems. But in order to 
develop models, the phenomenon being “modeled” first must 
be described. In short, description is the basis of all intelligence 
research. 

This chapter and the next discuss various methodologies 
that are or can be used in strategic intelligence production. This 
chapter addresses methodologies that are used for descriptive or 
analytic purposes. The next chapter will address methodologies 
that are used more commonly for predictive purposes even 
though they may be essentially descriptive. Some of the 
methodologies described in the next chapter relate to devel¬ 
oping conceptual models. Developing, validating, and applying 
these conceptual models to a variety of different problems 
qualifies as predictive research in the most rigorous sense. Other 
methodologies described in the next chapter may be equally 
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Figure XVII-1. Basic Analogy Model^ 


^The analogy models are based on the discussion presented by Max 
Black in Critical Thinking: An Introduction to Logic and Scientific 
Method (New York: Prentice-Hall, Inc., 1946), pp. 291-297. 


















































similar engines and if the performance characteristics of one 
engine were known, then the performance of the “unknown” 
aircraft could be inferred fairly reliably. The strength of this 
argument derives from the fact that the performance of the 
aircraft is logically connected to the quality of its engine. It is 
the ability to establish the cause-and-effect relationships or the 
invariant relationships between qualities and capabilities that 
ultimately determines the strength of analogies. 

In the case of inferring the capabilities of an “unknown” 
aircraft on the basis of associated qualities, the argument would 
look like Figure XVI1-2. 

The key to reasoning from analogy revolves around the 
bases that are used to establish similarities, The “force” of an 
argument from analogy is a function of how well connections 
(or “linking generalizations”) can be established. If the connec¬ 
tion between a given condition and a specified result is a strong 
one, then a good case can be made for a generalization drawn 
by analogy. 

But an important part of reasoning by analogy is to consider 
also those conditions, qualitites, characteristics, and so on, that 
are dissimilar between two phenomena. It may well be the case 
that the dissimilarities are so great that the similarities that do 
exist are negligible. 

Analogies are used in many different kinds of intelligence 
analyses from military and political to industrial intelligence. 
For example, major U.S. auto makers purchase their competi¬ 
tors’ models as soon as they appear in the showrooms. The new 
cars are taken to laboratories where they are completely and 
methodically disassembled. Each component, regardless of how 
seemingly insignificant it might be, is priced. The aggregate 
price for all of the parts collectively represents the material 
costs of producing one unit. To these costs would be added 
overhead, labor, and profit rates—rates which are generally 
similar throughout the industry. Reasoning by analogy, i.e., 
assuming that it would cost one producer the same amount to 
produce or purchase the same components used by another, the 
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Figure XVII-2. Strong Analogy Model 
























































major auto producers can estimate their competitors’ per-unit 
production costs, any cost-saving measures taken, and how 
much profit is earned by the sale of a single unit. 

The strategic intelligence counterpart of this example 
(which is also an analogy) is not hard to envision. What 
confounds the problem in intelligence production, however, is 
establishing currency equivalences. General Motors, Ford Motor 
Company, and Chrysler all use the same currency. However, 
determining the rate of exchange between th eyaun or the ruble 
and the U.S. dollar is a different matter. 

Analogies may be economic, (as in the example just 
described), physical (as was the case in the comparison of 
aircraft), or historical (as will be shown in the next chapter 
when Max von Hoffman predicted the behavior of two Russian 
generals on the basis of previous behaviors). Reasoning by 
analogy is fairly common in Economic Intelligence, in Scientific 
and Technical Intelligence, and, to a lesser degree, in Political, 
Biographic, and Armed Forces Intelligence. The technique is 
useful, but it must be used guardedly. In the excitement of 
discovering superficial similarities between phenomena, it is 
very easy to overlook the significant differences that may exist 
as well. 

Link Analysis 

Link analysis is an analytic technique for making relation¬ 
ships explicit. The technique was first used in human factors 
research but has since been applied to problems of social 
analysis as well. The technique requires the preparation of an 
association matrix and a link diagram based on the matrix. For 
example, based on a variety of intelligence reports, a matrix 
showing the confirmed and suspected associations of a number 
of individuals might be prepared as shown in Figure XVII-3. A 
solid dot represents a strong link (a confirmed association), and 
an open dot represents a suspected association. (No dot 
represents no known or suspected association.) 

Based on the matrix, a link diagram can be prepared (Figure 
XVI1-4). The solid lines represent strong (or confirmed) links; 
the broken lines indicate weak (or suspected links). No lines, of 
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Figure XVII-3. Association Matrix 


course, indicate that no association had yet been established. 
The link diagram can be enhanced further by superimposing 
organizational structures on the personnel linkages as shown in 
Figure XVI1-4. 

Link analysis can be used to construct inferential structures 
of organizations or interactions which can be tested later. It is 
very well suited for hypothesis construction and can be applied 
to a variety of problems in Armed Forces Intelligence (e.g., 
orders of battle), in Political Intelligence and in Sociological 
Intelligence research and analysis.^ 


^For more information, see Walter R. Harper and Douglas H. Harris, 
“The Application of Link Analysis to Police Intelligence,” Human Factors 
17 (April 1975): 157-164. 
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Figure XVII-4. Link Diagram with Organizations and Personnel 


252 


Sociometry 


In its applications to problems in social psychology, 
sociometry relates to techniques for depicting graphically the 
structure of relationships within a group. An organization chart 
may depict formal relationships among elements within an 
organization; however, organization charts do not reveal who 
actually talks to whom. Sociometric depictions—sociograms—do 
reveal the actual patterns of lines of communications and 
interaction. 

Given access to a group, interactions can be measured 
directly by observational techniques, or they can be measured 
by use of scaling devices such as questionnaires that would be 
administered to members of a group. However, the types of 
groups that are of primary interest to the intelligence analyst or 
researcher are generally groups to which the researcher has only 
limited access. Consequently, the more conventional measuring 
techniques cannot be used. 

One crude sociometric technique, however, might involve 
counting the number of instances in which a government 
official interacted with other officials at a formal reception. 
(But preparing a sociogram of this behavior would be of 
questionable value because a formal reception is an artificial 
situation and important people may talk to unimportant people 
as a matter of courtesy.) 

Another crude sociometric procedure would be to count the 
number of times certain names appeared in state-controlled 
mass media. Of course, the manner in which names are treated, 
i.e., positively or negatively, must also be taken into account. 
The assumption in this type of analysis is that there is a direct 
correlation between the number of times a name appears or is 
mentioned and its importance. 

In instances in which behavior can be observed—a researcher 
analyzing communication traffic, for example—records could be 
made of who interacts with whom and who initiates the 
interaction. Observations could be recorded as shown below in 
Figure XVI1-5. 
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Figure XVII-5. Observation Checklist Converted into a Sociogram 
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In Figure XVII-5, the observations are recorded initially as 
check marks placed at the matrix intersection of the sender and 
receiver. The check marks are converted to ones and zeroes in 
the second matrix. This representation is necessary if matrix 
multiplication is to be used to determine the number of indirect 
connections between various senders and receivers. In the 
example above, for instance, if the matrix were squared, then 
the ikth element of the new matrix (a' j^) would be: 


ik 


n 

= 2 


“ a ij a jk 
J=1 


In the example above, the element aj 2 =0xl+lx0+0x0 

+ 1 x 1 = 1. In other words, there is only one indirect 
connection between elements 1 and 2. (Element 4 is the link, 
e.g., l-*4->2. Inspection shows, however, that there is also one 
direct connection between elements 1 and 2.) 

This example, of course, is simple, and the linkages could be 
determined directly from the sociogram without too much 
difficulty. However, if the number of elements were great or if 
the analyst were concerned with determining various degrees of 
connectivity, then matrix multiplication would be necessary. 

Another crude sociometric technique is to assess the 
significance of an individual on the basis of his proximity to the 
leader. For example, for years Lin Piao, the former heir 
apparent to Mao Tse Tung, could be seen standing next to Mao 
on reviewing stands, and the vicissitudes of Chou En Lai could 
be observed (if not measured) by the number of other Chinese 
officials who would stand between Chou and Mao. The rank¬ 
ordering of Soviet and Chinese officials at affairs of state- 
parades, funerals, reception lines, and so on—has been used 
widely as a crude sociometric device for establishing the relative 
prestige and importance of the individuals. Literally and 
figuratively, in the Soviet Union and in the People’s Republic of 
China, people who fall from power fade out of the picture. 
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Sources that might be consulted for additional information 
about sociometry include Helen Hall Jennings, So dome try in 

Group Relations 4 5 6 and Leadership and Isolation A more recent 
work, and the work used as the basis for the example cited in 
this section, is James S. Coleman, Introduction to Mathematical 
Sociology. ^ 

Sociometric techniques can be used in Political Intelligence, 
in Sociological Intelligence, in Armed Forces Intelligence (e.g., 
orders of battle, traffic analysis), and conceivably in Biographic 
Intelligence. 

Game Theory 

Game theory is basically a theory of decision making by 
participants in a conflict or competitive situation. In a 
competitive situation, each participant attempts to “influence 
in the action” in such a manner that the outcome is an 
optimal one for him. Game theory permits the decision maker 
to select an optimal course of action in light of the number of 
options available to each player and in light of the “rules” by 
which the options can be exercised. An optimal course of action 
is one which maximizes the player’s probability of success and 
minimizes his probability of failure or loss. 

Games can be classified according to the number of players 
and according to the nature of the payoff. With respect to 
payoff, games are classified as either zero-sum games or non 
zero-sum games. A zero-sum game is one in which the outcome 
of the game is a “win” for one participant and a “loss” (of 
equal value) for the other. Translating game theory terminology 
to a real world c onflict situation, an unconditional surrender 

4 Helen Hall Jennings, Sociometry in Group Relations (Washington, 
D.C.: American Council on Education, 1948). 

5 -, Leadership and Isolation (New York: Longmans, Inc 

1943). 


6 James S. Coleman, Introduction to Mathematical Sociology (New 
York: The Free Press, 1964). 
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would be an example of a zero-sum conclusion, and a 
“negotiated settlement” would be an example of a non 
zero-sum outcome. 

Policy makers, disarmament negotiators, planners, and field 
commanders all use game theory in one form or another. For 
example, when a field commander makes his estimate of the 
situation and then chooses an appropriate course of action, he is 
using game theory. Although it is seldom done in the field, an 
estimate of the situation can be expressed mathematically in a 
series of matrices. The advantage of the mathematical depiction 
is that sometimes an optimal strategy becomes apparent that 
otherwise might be overlooked. 

A very basic example of strategies and payoffs is shown in 
the matrix below (Figure XVII-6). Blue’s strategies and payoffs 
are shown in the rows; Red’s strategies and losses are shown in 
the columns. In the example below, Strategy #1 for Blue could 
pay off 7 or 5, whereas Strategy #2 would yield 5 or only 3. 
From the standpoint of maximum gain, Strategy #1 would be 
better for Blue. But Red can influence the outcome as well. For 
example, if Red would select Strategy #1, Red could lose 7 or 
5. However, by selecting Strategy #2, Red could lose 5 or only 
3. Obviously, Strategy #2 would be better for Red. In Strategy 
#2, Red’s losses would never exceed 5, and they might be as 
low as 3. 

In this example, the optimal choice for both sides would be 
a combination of Blue’s Strategy #1 and Red’s Strategy #2. 
This combination will guarantee that Blue’s payoff will be at 
least 5, and it guarantees that Red’s losses will never exceed 5. 
When a single value exists that is at the same time the 
minimum value in a row, and the maximum value in a column, 
the value is called a saddle point. In conflict situations, saddle 
points are usually the places where negotiations would prove 
most profitable to both sides. 

If the payoff matrix in the example just cited contained 
other values (e.g., Figure XVII-7), no saddle point would exist. 
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Figure XVII-6. An Example of a Two-Person Game Matrix 


In a case such as this, each player should adopt a mixed 
strategy by using each strategy a certain percentage of the time 
in order to maximize his gains and minimize his losses. 

The percentage of the time a strategy should be used by a 
player is derived in the following manner: 1) each player takes 
the absolute difference between the payoffs associated with 
each strategy. For Blue, I 7 - 31 = 4, and 15-61 = 1. For Red, 

17 - 51 = 2,and 13 - 61 = 3. 2) The differences between payoffs 

are summed for each player. 3) Each player calculates a 
proportion derived from each strategy, and assigns that propor¬ 
tion to the alternate strategy. Therefore, Blue should play 
Strategy #1 one-fifth of the time and Strategy #2 four-fifths of 
the time. Red, on the other hand, should play Strategy #2 
three-fifths of the time, and Strategy #2 two-fifths of the time 
(Figure XVII-8). 
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RED 

STRATEGY #1 STRATEGY #2 


STRATEGY #1 7 3 

BLUE " 7 

STRATEGY #25 0 


Figure XV1I-7. An Example of a Two-Person Game Matrix with 
No Saddle Point 



Figure XV11-8. Payoffs with a Mixed Strategy 
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Assuming that Blue always wishes to maximize his gains and 
that Red always wishes to minimize his losses, the expected 
payoff for Blue would be obtained by multiplying the values in 
column 1 by the proportions of the time Blue played each 
strategy, or 

(7 x i) + (5x ±)= 22 
Column 2 gives the same result, i.e., 

(3 x ) + (6 x y ) = 22 

Likewise, for Red, the expected loss would be obtained 
similarly, e.g., 

(7 x |) + (3x J-)= 22 

and 

(5xy) + (6xy )=21 

Since this is a zero-sum game, Red’s losses always equal Blue’s 
gains in the long run. In a mixed strategy solution, each strategy 
must be chosen randomly if the calculated payoff is to result. 

To the reader who is unfamiliar with game theory, it may 
come as a surprise to discover how complex a seemingly simple 
game may become. For example, in the game of “Morra” in 
which each of two players shows 1, 2, or 3 fingers and at the 
same time guesses the number of fingers his opponent will 
show, each player has 9 choices of strategy. And the possible 
combinations of payoffs for both players is 81.7 

In gaming” a situation in which an aircraft loaded with a 
certain mix of electronic devices would be flown (symbolically) 


There are 9 strategies for each player: hence, a square matrix with 9 
rows, 9 columns, and 81 payoff squares. 
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against a target with another mix of electronic detectors and 
fire control devices, the combination of choices and payoffs 
(i.e., combinations which would optimize the chances of the 
aircraft’s reaching the target undetected) could be of such 
magnitude that a computer would be necessary in order to 
identify the optimal mix. It is for this reason that computers are 
usually used in major war gaming exercises. 

Game theory, as developed by John von Neumann and 
Oskar Morgenstern, tends to be highly mathematical and 
addresses conflict primarily in the economic sense. Since the 
publication of their Theory of Games and Economic Behavior , 8 * 
however, game theory has been applied to military and political 
conflicts as well. Perhaps the most popular and essentially 
nonmathematical treatment of game theory related to political 
and military problems can be found in Thomas C. Schelling’s 
The Strategy of Conflict2 

Other works that might be useful to the intelligence 
researcher include J. D. Williams, The Compleat Strategyst 10 
(an excellent book for the researcher unfamiliar with game 
theory); Melvin Dresher, Games of Strategy^ ; and J. C. C. 


8 John von Neumann and Oskar Morgenstern, Theory of Games and 
Economic Behavior (Princeton: Princeton University Press, 1947). 

^Thomas C. Schelling, The Strategy of Conflict (Cambridge, Massa¬ 
chusetts: Harvard University Press. 1963). 

D. Williams, The Compleat Strategyst: Being a Primer on the 
Theory? of Games (New York: McGraw-Hill Book Company, Inc., 1954). 

Melvin Dresher, Games of Strategyr. Theory and Applications 
(Englewood Cliffs, New Jersey: Prentice-Hall, Incl, 1961). 
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McKinsey, Introduction to the Theory of Games. 12 A short, 
humorous, but not at all trite book on game theory is John 
McDonald’s Strategy in Poker, Business and War . 13 

Game theory is basic to simulations involving conflict, e.g., 
war games. Although game theory is not a predictive technique 
per se, its application does permit the researcher to anticipate 
appropriate strategies for all players. Assuming that all players 
of the game were rational and that the players would opt for 
the minimax solution—the solution which minimizes the 
chances for loss and maximizes the chances for success—game 
theory can be used as a basis for predicting outcomes given 
certain conditions of conflict. 

War Gaming 

War games are experiments with alternative tactics and 
strategies in a conflict situation.^ Participants in war games 
often use game theory, but game theory and war gaming are not 
synonomous. Game theory relates to techniques of selecting 
optimum strategies. War gaming involves the applications of 
various strategies under various constraints in conflict situa¬ 
tions. Game theory, in short, provides the theoretical basis for 
war gaming. One can utilize game theory apart from war games, 
and war games can be played without a mathematical analysis 
of payoffs. 

1 hree terms are used typically in the discussion of war 
games: games, model, and simulations. Very briefly, a game is a 
contest played according to rules and decided by skill, strength, 

I2 J. C. C. McKinsey, Introduction to the Theory of Games (New 
York: McGraw-Hill Book Company, Inc., 1952). 

13 

John McDonald, Strategy in Poker, Business and War (New York: 
W. W. Norton & Company, Inc., 1950). 

14a 

As the term is used in this context, war games would include any 
type of conflict simulation. By this definition, simulations of political, 
economic, and technological conflict would be considered variations of war 
games as are the more familiar military conflict simulations or Kriegspiele. 
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or chance. Games are essentially symbolic conflicts. No one gets 
hurt in the symbolic conflicts, but lessons can be learned from 
these conflicts nevertheless. A model , on the other hand, is a 
representation of an object or a process. For example, a 
mockup is a model of a real piece of equipment, and a flow 
chart is a model of a process. Simulations , finally, are dynamic 
imitations of processes. To put it another way, a simulation is a 
dynamic model. 

War games can be classified according to the extent to 
which they are abstract. Army-wide games in which units 
actually maneuver on the terrain and engage each other 
represent one end of a continuum. Computer simulations of 
conflict represent the other end. In the middle of this 
continuum are any number of combinations of realistic and 
symbolic analogs. A command post exercise (CPX) may be 
considered a war game, but other than the command posts, no 
subordinate units move physically. The ‘‘enemy” in a war game 
may consist of human “Aggressor Force” personnel, or the 
enemy may consist simply of unit designators and numerical 
indications of strength given to the players in writing or 
displayed electronically on a cathode ray tube (CRT). 

The outcome of a game may be determined by the umpires 
on the ground who observe the action and ostensibly make 
objective judgments, or the outcome may be determined by the 
mathematical manipulation of variables according to some 
theoretical and, ideally, validated model. 

The nature of the game to be played determines the nature 
of the steps involved in the design of the game. For example, a 
game could be a man-to-man game which would involve the 
actual movement of forces; or it could be a computer-assisted 
game, probably played on maps, in which a computer per¬ 
formed the clerical duties associated with games—duties such as 
scoring, determining kills, and keeping time. Or the game could 
be a man-machine game in which the machine made all of the 
evaluations and determined outcomes on the basis of the values 
submitted, tactics or strategies used, and on the basis of the 
rules of play which were programmed into the computer. 
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In the first type of game, a scenario could be prepared and 
given to the players. The scenario would describe the general 
situation, perhaps a general order of battle, the respective 
missions of the two (or more) sides, and the rules of play. Upon 
receipt of the general plan, the forces would move to the field. 
Umpires who accompanied the opposing sides would do the 
scoring, make kill determinations, and perform other book¬ 
keeping functions manually. The computer-assisted and man- 
-machine games, on the other hand, would require the prepara¬ 
tion of a series of mathematical or logical models which would 
represent certain sub-elements of the conflict; for example, a 
model of a surface-to-air-missile engaging an aircraft, or an 
antisubmarine warfare model, and so on. Developing these kinds 
of games requires a sequence of steps which include 1) defining 
the action to be gained; 2) identifying and classifying variables 
in terms of significance; 3) identifying and quantifying relations 
among variables; 4) designing the model (initially in flow chart 
form) and translating the model into computer instructions; 5) 
specifying tradeoffs for the models and programming the 
computer accordingly; and 6) collecting real-world data 
for the purposes of obtaining realistic values for variables and 
for validation purposes (i.e., for determining if the simulated 
conflict did in fact yield outcomes that a genuine conflict 
situation would yield). 

War games involving international conflict may require 
months, if not years, of preparation in order to develop models 
that would address the numerous sub-elements involved in such 
a conflict; for example, the military, political, and economic 
factors and their major interactions. Typically, as war games 
become “more global,” they also tend to become more general 
simply because the costs in modelling anything less than a major 
element of the conflict in fairly general terms would become 
prohibitively expensive. Furthermore, the nature of the game 
play also tends to become more complex when numerous 
sub-elements are addressed. 

With respect to the predictive value of war games, the point 
made earlier in the discussion of game theory applies: a war 
game can only determine the statistical probability of an event’s 
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occurring given certain variables and rules for manipulating 
these variables. 15 Games serve an excellent diagnostic purpose 
by identifying which variables have the greatest effects on the 
outcomes, however. Aside from their values as training mecha¬ 
nisms and as a research vehicle, games are usually used to 
evaluate various courses of action in order to arrive at 
conclusions with respect to plans and policies. 

An excellent overview of war gaming is presented in Clark 
C. Abt’s article, “War Gaming,” in International Science and 
Technology . 16 Other sources include Melvin Dresher’s Games 
of Strategy , 17 R. Duncan Luce’s and Howard Raiffa’s Games 
and Decisions, 1 ^ and Thomas C. Schelling’s The Strategy of 
Conflict , *9 mentioned earlier in the discussion of game theory. 
Personnel of the Studies, Analysis and Gaming Agency, Office 
of the Joint Chiefs of Staff, could also describe specific types 
and applications of games used for joint planning purposes. 

15 One of the authors recalls observing a week-long war game played 
at the Naval War College in which the outcome of an engagement 
depended upon “Green’s” forces detecting “Red’s” surface units. A 
reconnaissance flight failed to detect the presence of “Red’s” forces, and 
in the ensuing battle, “Green” was “defeated.” Having access to both the 
“Red” and the “Green” side as well as to the master display (which the 
players did not), the author noted that the success of that single, critical 
flight was determined by one run of an analog computer. That single 
computer run (or toss of the dice) would hardly permit anyone to state 
unequivocally that “Red’s” deployment and tactics were necessarily better 
than “Green’s,” but it did drive home dramatically the importance of that 
single reconnaissance mission. 

^Clark C. Abt, “War Gaming.” International Science and Technology 
(August 1964): 29-37. 

17 Dresher, Games of Strategy. 

!8r. Duncan Luce and Howard Raiffa, Games and Decisions (New 
York: John Wiley & Sons, Inc., 1957). 


^Schelling, Strategy of Conflict. 
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Linear Programming 


Goals can sometimes be expressed in terms of achieving a 
maximum payoff for a minimum investment or risk. For 
example, an investor might want to receive the maximum 
number of dollars return on his investment; a manufacturer 
might want to produce the maximum number of units with a 
minimum amount of material, and a purchaser might want to 
buy the most or the best of a commodity for the least amount 
of money. Linear programming is a method which allows one to 
find a maximum or a minimum point (optimum point) to 
satisfy a goal (objective). 

I " n ! arl .y every tyP e °f operation (or organization), there 
are limitations or constraints upon the use of time and 
resources. If the limitations can each be expressed by the 
equation of a line, or 


y = ax + b (]) 

then linear programming may be used to find an optimum 
solution. 

For example, the operations officer of a PSYOP unit that 
was operating a strategic radio broadcasting facility could use 
linear programming to maximize the number of potential 
listeners in light of certain operational constraints. His basic 
objective would be to maximize the size of the target audience 
mat, would hear spoken commentary since the PSYOP “mes¬ 
sage” would be transmitted in this commentary. However he 
realizes that the best way to gain and to hold his target 
audience s attention is to play traditional and popular music. 

Commentary, therefore, would have to be interspersed between 
musical selections. 

If the broadcast day were defined as a mixture of time 
evoted to music and commentary, the proper proportions of 
each type of broadcasting can be discovered by solving a simple 
linear programming problem. 
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Initially, the operations officer notes that his broadcast day 
is 12 hours long and that it will be divided into two types of 
broadcasts. The broadcast day is described by the linear 
equation 


12 > x + y (2) 

where x equals the number of hours per day devoted to 
commentary, and y equals the hours devoted to music (Figure 
XVII-9). 

The next limitations placed upon the program day are that 
there must be no fewer than 5 but no more than 10 minutes per 
hour reserved for commentary. These limitations must be 
converted to the same units (hours) in order to be graphed. The 
two equations to describe these limitations are 


X > 1 

(3) 

y < 2 

(4) 


since 5 minutes per hour equals 1 hour per 12-hour day, and 10 
minutes per hour equals 2 hours per 12-hour day. 

Finally, the number of man-hours available to prepare the 
broadcasts is limited. There are 18 hours of labor available each 
day. It only takes 1 hour to prepare 1 hour of music, but it 
takes 5 hours to prepare 1 hour of commentary (actually, the 
preparation of commentary consumes about 15 hours for each 
hour of broadcast time, but it can be assumed that each message 
is repeated 3 times). The equation to describe the labor 
constraint is 


18 > 5 x + y (5) 

After all of the important constraints are specified, the 
original objective must be expressed in terms of the common 
units (hours in this case). Since the objective is to maximize the 
number of listeners to the station, something must be known 
about how different kinds of programs attract listeners. In the 
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Figure XVII-9. Graph of Optimal Programming with Imposed 
Constraints 


current example, it is assumed that music is 20l times as likely to 
attract and hold listeners as commentary.- 0 Therefore, the total 
number of listeners would be a function of the number of 
commentary hours, plus 20 times the number of music hours, 

times a constant, or 


N = k(x + 20y) 


( 6 ) 


The only points which are permissible solutions for the 
objective function are those which fall within the boundaries of 
all the constraints. For example, points B, C,.andID as wehasal 
the points in the shaded polygon (Figure XVII-9) are^owa 
mixtures of music and commentary programming Point C on 
the line x + y = 12 which means it uses up all 12 hours ot 
me line y 5 X + y = 18, which means it uses 

IJSTiXJ onli!wE C also falls between «he limits of 
5 and 10 minutes of commentary per hour. 

Point A, on the other hand, is not a valid solution because it 
allows no time for commentary and, therefore, violates the 
constraint of x > 1. It does satisfy the other two constraints 
an5 would be a valid solution if the x > 1 constraint were 
abandoned. 

Linear programming theory also states that optimal[ solu¬ 
tions for the objective function are found at a corner or the 
polygon of acceptable solutions. The five comers of the 
polygon in the current example are B, C, D, b, and b ine omy 
way to determine win ch corner is (he optimal solutton is to 
™iv P pmiatinn (6) at each comer (Table XVI1-1). 


20 This ratio is not unrealistic. In countries where listening to foreign 
iroadcasts is prohibited, radio programming which utilizes the maximum 
mount of music for the minimum amount of spoken commentary poses 

^ .1 _lirfnnnr 
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Table XVII-1. X and Y Values of Five Points 


POINT 

x VALUE 

y VALUE 

K(x + 20y) = N 

B 

1 

1 1 

221 K 

C 

1.5 

10 5 

211.5K 

D 

2 

8 

16 2 K 

E 

1 

0 

IK 

F 

2 

0 

2K 


Clearly, the value of N is maximized at point B (N=221K). 
Therefore, the maximum number of listeners would be achieved 
when the program consisted of 1 hour of commentary, or 5 
minutes per hour, and 11 hours of music. Point B solution, 
although it uses the entire 12-hour broadcast day, does not 
require the 18 hours preparation time. In fact, it requires 
(equation 7) only 


5(1) 4-11 = 16 


(7) 


16 hours of labor. 

The example of radio broadcasting contained in this 
explanation is much simpler than most real world linear 
programming problems, but it contains all the necessary 
elements. In summary, the following factors must be present if 
linear programming is to be applied to a problem: 

1) The researcher must be trying to optimize (maximize 
or minimize) something 

2) There must be more than one type of product, 
commodity, or output under consideration. In this case, the 
product was the composition of a broadcast day 

3) There must be constraints or limitations upon the 
resources which must be used to produce the commodity or 
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product. (In the example cited, there were labor constraints, a 
limit on the broadcast day, and the constraint that between 1 
and 2 hours must be used for commentary during a broadcast 
day.) 

4) All constraints must be able to be described by linear 
equations 

The current example involved only two products. But linear 
programming may be applied to solve for any number of 
products. Most linear programming problems are complicated 
and time-consuming; consequently, they are best solved by use 
of computers. The General Electric System of the Defense 
Intelligence School has computer programs for solving linear 
programming problems. 

There are some optimization problems that have con¬ 
straints that cannot be expressed by lines, but which should be 
described instead by curves. Recently, mathematical program¬ 
ming techniques have been developed to handle such problems. 

More detailed explanations of the applications of linear 
programming may be found in Great Ideas of Operations 
Research ,21 by Jagjit Singh and Introduction to Finite Mathe¬ 
matics ,22 by John G. Kemeny and others. A more advanced 
source is Linear Programming and Economic Analysis ,23 by 
Robert Dorfman and others. 


2 i Jagjit Singh, Great Ideas of Operations Research (New York: Dover 
Publications, 1968), pp. 87-102. 

2“John G. Kemeny, J. Laurie Snell, and Gerald L. Thompson, 
Introduction to Finite Mathematics (Englewood Cliffs, New Jersey: 
Prentice-Hall, Inc., 1956), pp. 249-266. 

^Robert Dorfman, Paul A. Samuelson, and Robert M. Solow, Linear 
Programming and Economic Analysis (New York: McGraw-Hill Book 
Company, Inc., 1958). 
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Table XVII-2. Number of Acts of Violence and Monthly Rates of 
Inflation 


MONTH 

MONTHLY RATE OF 
INFLATION 

(*l> 

NUMBER OF ACTS OF 
VIOLENCE/MONTH 

(V|) 

RATE 

X ACTS 

(x . . y .) 

1 3 1 

RATE 

SQUARED 

(*i y,) 

ACTS 

SQUARED 

(y j) 2 

1 

4 

2 

,8 

, 16 

4 

2 

106 

6 

9.6 

2.56 

36 

3 

2 „ 4 

14 

33.6 

5 0 76 

196 

4 

1.6 

17 

27.2 

2.56 

289 

5 

3.2 

14 

44.8 

10.24 

196 

6 

3.6 

26 

93.6 

12.96 

676 

7 

3.2 

22 

70.4 

10.24 

484 

8 

4.4 

29 

127.6 

19.36 

841 

9 

4.0 

21 

84.0 

16.00 

441 

10 

1.2 

7 

8.4 

1 .44 

49 


2*. = 25.6 

2y. = 158 

Sx.y. = 500 

1 3 1 

£x. 2 = 81.28 

£y, 2 = 3212 


/U. ux. = 2,56 

fiuy . = 15 8 
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straight line between the points so that the line would be as 
dose as possible to each point on the graph. However, there are 
disadvantages with this technique: it is subjective, and unreli¬ 
able. Different analysts will obtain different results using this 
method, and, therefore, it provides a poor foundation for 
further analysis of the strength of a relationship. 

The most common mathematical method for drawing a 
regression line is called th z least-squares method. The method is 
so named because it specifies the line which makes the total of 
the squared distances from actual y values to the line as small as 
possible. The actual y i minus the predicted y (yj - yp Figure 
XVII-10) is measured parallel to the y axis. 

The final form of the equation is the same as the equation 
of any line, namely: 


y - ax + b (1) 

where Y is a predicted Y value, X is an actual X value, a is the 
slope (or rate of increase in Y divided by the rate of increase in 
X), and b is the intercept point (or where the line crosses the Y 
axis). 

The equations necessary to solve for the values of a and b 

are: 

10 10 

2 y. = a 2 x- + b • n (2) 

i=l 1 i=l 1 

and 

10 10 - 10 

2 x-y- = a 2 X; + b 2 x- (3) 

i=l 11 i=l 1 i=l 1 

Although equations (2) and (3) may appear involved, they 
are easy to solve. The analyst knows all the x^ values and y- 
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values, and he knows that n equals 10 (i.e., 10 months, in this 
example). He can compute the necessary sums and products 
(which appear in Table XVII-2). Therefore, (2) and (3) become 
two simultaneous equations containing two unknowns which 
can be solved by elementary algebra. Making the necessary 
substitutions, 


158 = 25.6a + 10b (4) 

and 

500 = 81.28a + 25.6b (5) 

The value for a equals 6.067. The value for b equals .268. 
The equation for the regression line is: 

y = 6.067x + .268 (6) 

When the analyst has an x value, he then uses this equation to 
predict a corresponding y value. If he were told, for example, 
that next month’s inflation rate would be about 2.8, he would 
compute 


y = (6.067 x 2.8) + .268 (7) 

y = 17.2556 (8) 

When the analysis uses the equation y = ax + b, the purpose 
is to predict y from a given x value. If it were desired to predict 
x from a given y value, the proper equation would be x = ay + 
b. The a and b values in this equation would be different from 
the a and b values in y = ax + b. 

Throughout this discussion, the prediction of y values from 
x values has been stressed. The prediction of one variable from 
another is very different from saying that one variable causes 
another. Usually several analyses of the relationships among 
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variables would be necessary before the cause of a variable 
could be established. 24 

After the relationship described by the equation of a line is 
established, the analyst may wish to determine how well the 
line describes the set of data points. If the line contained all of 
(lie actual data points, the points are said to be correlated 
perfectly, and the correlation coefficient for the data would 
equal ±1.0. At the opposite extreme is a set of points which 
would appear as a fat cloud when plotted. Such a set of data 
points would have a correlation coefficient of 0.0. A correlation 
coefficient is positive if one variable increases while the other 
variable increases. A correlation coefficient is negative if one 
variable increases while the other decreases. Either correlation 
may be important. 

The correlation coefficient is calculated using the following 
formula: 


nXxy - 2x2y 


/ 


(9) 


[n(2x 2 ) - (2x) 2 ] 


[n(2y 2 ) - (2y) 2 ] 


Again, equation (9) looks complicated. But it is easy to 
substitute the various products and sums computed earlier. For 
the current example, 



5000 - 4044.8 

(10) 

r = 


v/ 

(157.44) (7156) 


8999 or approximately .90 

(ID 


24 See also the discussion of causality in Chapter XIV, “Foundations 
of Analysis: Some Basic Concepts.” 














A correlation coefficient of .90 is very high, as is indicated by 
the fact that none of the points is very far from the regression 
line (Figure XVII-10). 

If the analyst wished to establish a confidence level for an 
estimated value of y, he must then compute the standard error 
of the estimate (Oy. x ). Since there is a normal distribution of 
data points around the regression line, it can be said that 68 
percent of the actual y values fall within lcry. x above or below 
the regression line. 

The formula for the Oy. x is: 


o 


y.x 



- bZy - aSxy 


n - 2 


( 12 ) 


Therefore, in the current example: 


\J32\2 - 42.344 - 3033.5 


\ /l 36.156 

8 


= 4.125 


(13) 

(14) 

(15) 


If the analyst were asked to predict the likely number of 
acts of violence, given that next month’s inflation rate would be 
about 2.8 percent, he would first compute equation (6) and 
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determine that there will be 17 acts of violence. Finally, using 
the o-y. x , he could report that there is a 68 percent likelihood 
that there will be 


17.2556 - lo, v = 13.1306 (16) 

y • X 

17.2556 + 1 a = 21.3806 (17) 

y • X 


or 13 to 21 acts of violence in the next month, and a 95 percent 
likelihood that there will be 


17.2556 - 1.96 a 7 = 9.1706 (18) 

y * x 

17.2556 + 1.96 a y# x = 25.3406 (19) 


or between 9 and 25 acts of violence in the next month. 

History has shown that citizen unrest is often correlated 
with economic instability, so the current example probably 
shows a meaningful correlation between sets of data. However, 
sometimes one discovers high values of correlation which are 
spurious. 25 The analyst’s best defense against placing too much 
importance upon the value of the correlation coefficient is his 
own knowledge of the variables. 

Although a line was fitted to the data sets in the current 
regression example, sometimes a curve rather than a straight line 
better describes a relationship. The mathematical expressions 
necessary to create these curves are, for the most part, much 
more complicated than the equations for the linear relationship. 

The current example used only two variables, but regression 
and correlation analysis may also be completed for three or 


25 For example, for many years the number of graduates in theology 
from Oxford University was nearly perfectly correlated with the number 
of arrests made for prostitution in an Australian city. 
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more variables. For example, the monthly rate of inflation, the 
number of acts of violence per month, and the margin of profit 
for large industries could all be subjected to correlation analysis. 

Regression analysis and correlation analysis are described in 
nearly every book on statistics. Two books that might be 
consulted are George H. Weinberg and John A. Schumaker, 
Statistics: An Intuitive Approach ^6 and William L. Hays, 
Statistics .27 

Graphic Depictions as Aids to 
Analysis 

The value of graphics in printed reports or as briefing aids is 
obvious. What may not be apparent, however, is the value that 
graphics may have in the analysis of data. Graphic depictions do 
more than transform numerical data or words into pictures. 
Very often graphic depictions reveal relationships that might 
not be apparent otherwise. Described below are four examples 
of the uses of graphics in various types of analyses. 

Utility curves 

One example of a graphic depiction that directly aids 
analysis is the utility curve. Utility curves are often used in 
business and industry to determine ‘'break even” points-points 
where, for example, a return could be realized on an invest¬ 
ment, or the point in time when the per-unit cost of 
manufacturing a certain commodity would be lowest. In mass 
production, the first item produced is the most expensive. As 
more items are produced, the per-unit cost decreases until a 
certain point is reached. Utility curves help the analyst to 
determine these points (Figure XVII-11). 


^George H. Weinberg and John A. Schumaker, Statistics: An 
Intuitive Approach , Second Edition (Belmont, California: Brooks/Cole 
Publishing Company, Inc., 1969). 

^William L. Hays, Statistics (New York: Holt, Rinehart and Winston, 
1963). 
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UNIT 
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Figure XVII-11. A Utility Curve Showing the Decreasing Costs Per 

Unit Over Time 


Utility curves might be employed in analyzing geo-political 
problems as well. For example, the U.S., over the years, may 
have been providing a fixed amount of assistance annually to a 
country in exchange for base privileges. It may be the case, 
however, that the recipient country would begin restricting 
privileges granted earlier. A graph such as the one shown in 
Figure XVII-12 would depict the value (or worth) of the 
privileges received for a fixed amount of dollars paid out over a 
number of years. The worth or value of the privileges received 
would have to be expressed in a unit that would permit 
comparisons with expenditures. In short, it might require the 
researcher or analyst to place a dollar figure, for example, on 
intangibles such as “international good will,” or on the worth of 
a country participating in an alliance -admittedly, no easy 
matter in some cases. 


281 


















Figure XVII-12. A Graph of the Actual and Projected Worth of a 
U.S. Expenditure in Country “X” Over Time 
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The graph would indicate that the value of the privileges 
gained versus the funds expended fell off sharply after the 
second year, and that the projected return on investment was 
estimated to continue to decline but at a slower rate. The 
graphic depiction, in this case, makes the presentation of data 
more dramatic than, say, a column of figures. But the graph 
may serve another purpose as well. 

If the political or military objectives of the host country 
were known, a similar type of curve could be plotted for that 
country. For example, the host country might need a certain 
quantity of materiel to equip a specified number of units of its 
armed forces. If the major source of financial aid used for the 
purchase of this equipment were credits or payments made by 
(he U.S. for base privileges, then the degree to which that 
country was progressing toward its immediate objective could 
be plotted. (Figure XVII-13.) 

The graph shows that the point where the two lines cross 
over is about the point in time where the host country’s returns 
(returns in the form of some capability that was translated into 
dollars or into some other expression of utility) begin to 
accelerate. The crossover point might also represent a point 
beyond which the U.S. might no longer consider its investment 
to be cost effective. At the minimum, however, this crossover 
point would indicate a point in time when negotiations might 
prove most fruitful.^ 8 

In this example, only one “utile” value was expressed. 
Presumably, this single value represented a composite of many 
separate elements. Generally, it would be easier and perhaps 
more valid to plot the utilities of each component value 
separately rather than expressing all values as one single value. 


“^In game theory terminology, this crossover point would be roughly 
analogous to a saddle point. 
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Bar graphs for comparison 

As mentioned earlier, sometimes depicting data graphically 
reveals information that would not be too apparent otherwise. 
An actual example of this is shown below. The thematic output 
of North Korea’s international and domestic broadcast facilities 
was tabulated over a two-week period in 1971. The tabulated 
data, categorized by themes, are shown in Table XVII-3. 
Graphing the same data, however, made readily apparent the 
differences in the amount of time devoted to certain themes in 
domestic broadcasts versus the proportion of time devoted to 
the same themes in international broadcasts. For example, the 
theme “Propagation of Ideology,” constituted 34 percent of the 
output for international audiences, but only 14 percent for 
domestic audiences. On the other hand, the theme, “Present 
Success of Socialism,” constituted 49 percent of the output for 
domestic audiences, but only 18 percent for the international 
audience (Figure XVII-14). 

Terrain profiling 

Terrain profiling is often used to determine areas that could 
not be reached by direct fire weapons. But it can be used for 
other purposes whenever line of sight is critical. For example, a 
telecommunications analyst might want to determine how 
many microwave relay towers would have to be constructed in 
order to permit microwave transmissions to and from two cities. 
In the example shown in Figure XVII-15, for instance, the 
analyst could determine that seven relay towers would be 
required between Ban Pak and Hua Hin. His determination 
would be based upon a maximum transmission range of 
approximately 90 kilometers between towers and a direct line 
of sight requirement. The exaggerated terrain profile was 
prepared by extending vertical lines from the most plausible 
route for the location of towers—the route that would utilize 
the highest terrain elevations and would have the minimum 
“masking.” 
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Table XVTI-3. Comparison of North Korea’s International and 
Domestic Thematic Coverage for a Two-Week 
Period 


THEME 

INTERNATIONAL 

BROADCASTS 

DOMESTIC 

BROADCASTS 

MINUTES 

PERCENT 

MINUTES 

PERCENT 

PRAISE OF THE LEADER 

35 

3 

45 

3 

PROPAGATION OF IDEOLOGY 

349 

34 

177 

14 

GLORIFICATION OF THE 
REVOLUTION 

22 

2 

1 

1 

PRESENT SUCCESS OF 
SOCIALISM 

180 

18 

640 

49 

GUIDANCE FOR FUTURE 
SUCCESS 

12 

1 

41 

3 

INTERNATIONAL PRESTIGE 

210 

20 

191 

15 

SUPPORT FOR PEOPLE'S 
STRUGGLE 

8 

1 

32 

2 

NEGATIVE 

TREATMENT: 

U,S, 

32 

3 

12 

1 

ROK 

136 

13 

140 

11 

JAPAN 

46 

5 

16 

1 
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Figure XVII 14. A Bar Graph of Thematic Coverage of North 
Korean Domestic and International Broadcast 
Facilities for a Two-Week Period 
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Figure XVII-15. An Example of a Terrain Profile 
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Trend analysis 

Graphs are especially useful in trend analysis. 29 For 
example, the graph in Figure XVII-16 shows the number of 
anti-U.S. statements appearing in a foreign country’s official 
newspaper for an eight-month period. The graph reveals peak 
periods in May and June and plausible reasons for these peaks 
would have to be established, possibly from a chronology of 
past events. However, the analyst or researcher might be called 
upon to determine if the plotted data indicated anything 
unusual or significant. If prior data for similar periods in the 
past were available, separate graphs would be overlaid (or a 
combined graph prepared) as shown in Figure XVII-17, and a 
comparison could be made directly. In the example cited, for 
instance, it would appear that the current “trend” represented 
no dramatic departure from prior “trends.” 

Graphs can be used to plot likely times to completion. For 
example, if the rate of progress had been established for 
constructing the rail line from Maoming (21 41 N/110 51 E) to 
Hsinhsing (22 42N/112 13 E), then the estimated distances to 
be completed within certain time periods could be graphed 
(Figure XVII-18). Another way of depicting progress would be 
to make the estimated time plots on a quasi-map graph, as 
shown in Figure XVII-19. 

With respect to graphic depictions for analysis, an inexpen¬ 
sive book that an analyst or a researcher may find especially 
useful is Henning Nelms’ Thinking with a Pencil?® For those 
who would be interested in graphics as presentation modes 
rather than analytical tools, any number of books could be 
used, for example, Mary Eleanor Spear’s book, Charting 
Statistics . 21 For some novel, non-traditional ways of presenting 

29 See also the discussion of extrapolation, Chapter XVIII. 


^Henning Nelms, Thinking with a Pencil (New York: Barnes & 
Noble, Inc., 1964), p. vii. 

2 * Mary Eleanor Spear, Charting Statistics (New York: McGraw-Hill 
Book Company, Inc., 1952). 
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Figure XVII-16. Number of Anti-U.S. Statements Appearing in a 

Foreign Medium for an Eight-Month Period 



to 

vo 



Figure XVII-17. Number of Anti-U.S. Statements Appearing in a 

Foreign Medium for an Eight-Month Period. (Three 
Years) 














































292 293 




























statistical data, Roberto Bachi’s Graphical Rational Patterns: A 
New Approach to Graphical Presentation of Statistics 32 should 
be consulted. 

Summary 

^Descriptive methodologies are basic to descriptive as well as 
to predictive research. Very often it is necessary to describe 
what exists at the present time in order to be able to predict 
what will exist or occur in the future. 

>The construction of analogies is a basic analytical procedure 
used most universally in intelligence research. War games, 
simulations, and models are all analogies of different aspects of 
reality. Analogies are used as guides to reasoning in all of the 
components of strategic intelligence. 

> Graphic depictions are not so much specific analytical 
techniques as they are methods for transforming data. Conse¬ 
quently, they are relevant to descriptive as well as to predictive 
research and analysis activities. 


Roberto Bachi, Graphical Rational Patterns: A New Approach to 

Graphicat Presentation of Statistics (New York: Israel Universities Press 
1968). 
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CHAPTER XVIII 

PREDICTION, FORECASTING, AND HARUSPICY 


“What say the augurers?” 

“They would not have you to stir forth to-day. 

Plucking the entrails of an offering forth, 

They could not find a heart within the beast. ” 

Julius Ceasar, act ii, sc. ii 

With the intelligence community’s overriding concern for 
reducing uncertainty, it is not surprising that techniques for 
anticipating future events or conditions would have a special 
attraction for the intelligence researcher or analyst. This chapter 
discusses selected techniques for anticipating future events or 
conditions that have been (or can be) used in intelligence 
research and analysis. The first part of this chapter discusses the 
nature of prediction and forecasting, discusses assumptions 
underlying prediction, and describes the relationship of a 
technique to the nature of the phenomenon being considered. 

Although this chapter is concerned with methodologies for 
forecasting, there is nothing sacrosanct about the methodology 
which limits its application to predicting or anticipating future 
events. Simulations, for example, can be used for diagnostic 
purposes as well as for predictive purposes. That accurate 
predictions have been made on the basis of “non-predictive” 
techniques, as such, is not only possible, but is also highly 
probable. 

Types of Phenomena and their 
Relationship to Forecasting 

At the outset it should be apparent that some types of 
phenomena are easier to forecast than others. At one end of a 
continuum, for example, are short-term, unique events which 
occur at specific points in time. These are the most difficult 
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types of phenomena to predict because they may not have any 
precedents. Very often, unfortunately, these are the types of 
events that are of most concern to the intelligence community. 
An example of this type of phenomenon would be an 
unexpected coup that toppled a government overnight, or the 
defection of a key scientist on a highly classified defense-related 
project, or a discovery of large deposits of a strategic com¬ 
modity. 

In the middle of the continuum are events which occur over 
a period of time. There may be little doubt about their 
outcome, but the concern may be to establish when the 
outcome will occur or what the effects may be. Admittedly, the 
point in time at which a phenomenon started may be 
considered a unique event, but the event is nevertheless 
continuous for a given period of time. An example of a 
continuous event would be a country’s construction of its first 
aircraft carrier. The precise time that the carrier was started 
(e.g., the specific point in time when the first line was drawn on 
a plan) was a unique event. But, for the sake of this discussion, 
the time events and occurrences that would take place until the 
carrier completed its sea trials and became fully operational 
could be considered as a continuous sequence of events. 

At the other end of the continuum of phenomena are 
cyclical events or conditions whose occurrences are just short of 
inevitable; for example: phases of the moon, tides, seasons, 
and, of course, the diurnal cycle itself. 

Probabilistic Statements and their 

Relationship to Types of 

Phenomena 

Predictions in intelligence are nearly always couched in 
some probabilistic terms. (Even this statement is translatable 
into probabilistic terms.) Not surprisingly, the highest prob¬ 
abilities of occurrence can be associated with cyclical 
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phenomena. Not only is the probability of a cyclical event’s 
occurring high, but the confidence that an analyst or a 
researcher may have in his prediction is also high. 1 

With respect to continuous phenomena, once they have 
been detected, subsequent events or effects can often be 
predicted with a fairly high degree of accuracy. In many cases, 
precedents exist for continuous phenomena, and researchers can 
infer future events on the basis of analogous antecedents. 
Furthermore, by monitoring development on continuous 
phenomena, analysts and researchers can revise their estimates 
and change their probabilities. 

Unique events, however, pose the most difficult problems in 
forecasting. And it is in the realm of forecasting unique events 
that the more esoteric prediction methodologies are employed. 
Interestingly, as will be pointed out later, these methodologies 
for addressing unique events are essentially techniques for 
employing subjective judgments of experts. However, even these 
more esoteric techniques deal more often with possibilities 
relating to classes of events rather than probabilities relating to 
a specific, unique event. 

Assumptions—Pragmatic and 
Problematic 

Every prediction (or predictive model) is based on assump¬ 
tions. And it is usually the validity of the assumption that 
determines the accuracy of the prediction. 2 Perhaps one of the 


^As anyone who has worked with subjective probabilities can point 
out, the two expressions of likelihood are not the same. For every flip of a 
coin, the probability of tossing a “tail” remains .5 (50/50); but a gambler 
who has tossed ten consecutive tails may feel very strongly that the next 
toss will be a “head.” 

2 Sometimes the assumptions are valid, but the facts are wrong. For 
example, the assumption that the Japanese Manchurian Army would fight 
tenaciously was a valid one. But for years, the Japanese Manchurian Army 
existed only in the minds of U.S. military planners. 
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most fundamental assumptions is that events which occurred in 
the past will occur again in the future. The causal factors for an 
event’s occurring may not be known, but predictions based on 
the assumption that previous events will occur again are usually 
based on large numbers of observations. For example, it is not 
necessary to understand how or why lunar eclipses occur in 
order to predict that they will occur and when they will occur. 

Higher on an imaginary ladder of abstraction are assump¬ 
tions that the same collective factors that brought about (or 
caused) an event in the past will bring about a similar result in. 
the future. The most familiar expressions of this assumption are 
weather forecasts. Having established and tested cause-and- 
effect relationships and having classified and coded all of the 
factors that determine meteorological conditions, meteorologists 
can make very accurate predictions for different time periods.^ 
Predictions based on the identification of causal factors are 
generally highly reliable, and in laboratory situations, they are 
practically infallible. The identification of causal factors permits 
researchers to construct deterministic models—models which 
will unerringly predict accurate outcomes if the data provided 
to the model are correct. Again, this is relatively easy to do in 
laboratory situations, but very difficult to do in unstructured 
situations involving human behavior. 

Another common assumption, and one that manifests itself 
especially in the behavior of investors in stocks, is that a trend 
that exists now will continue—at least for the immediate future. 
Thus, upward surges of the market are stimulated in part by 
their own momentum, and downward plunges are exacerbated 
by the expectation that the downward trend will continue. A 
drop in the market generates pessimism which stimulates more 
selling. Aside from “technical adjustments” due to profit 
taking, trends continue until some potentially significant event 
brings about a change in buyers’ expectations. The assumption 


^That the predictions are sometimes inaccurate is due to a lack of the 
most recent information, not to a fallacy of the predictive models. 
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that trends continue (or will continue for some specified period 
of time) is the assumption that underlies all extrapolations , one 
of the more common techniques for predicting trends. 

In the realm of social behavior, assumptions are made about 
how the other side will behave. In fact, interpersonal relations 
would be impossible without minimal expectations (assump¬ 
tions) of how humans would behave under certain circum¬ 
stances. 

Expectations of human behavior are also based on observa¬ 
tions of past behavior, and again the prevailing assumption is 
that behaviors which existed in the past will persist in the 
future. 

This assumption paid off well in the decisive battle of 
Tannenberg in 1914. Very briefly, Colonel General Max von 
Prittwitz und Graffon’s Eighth German Army under Hinden- 
burg faced two Russian armies in East Prussia whose combined 
forces greatly outnumbered his own. However, Lieutenant 
Colonel Max von Hoffman, a German staff officer, recalled that 
the two Russian generals, Pavel Rennenkampf and Aleksandr 
Samsonov, were bitter enemies. (They were seen fighting in a 
railway station in Mukden during the Russo-Japanese War.) On 
the basis of their past behavior, von Hoffman predicted that 
neither would make any special effort to come to the aid of the 
other if he were hard pressed. On the basis of this prediction, 
Hindenburg ordered Prittwitz und Graffon to expose his flank 
and to engage the Russian Second Army commanded by 
Samsonov. Rennenkampf, true to his predicted behavior, did 
little to relieve the pressure on Samsonov who was subsequently 
defeated. Shortly thereafter, Rennenkampfs First Army was 
also defeated at the Battle of the Masurian Lakes. Von 
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Hoffman’s prediction of behavior based on past observation 
proved accurate. 4 

A fundamental assumption that is typically made about 
other humans, e.g., the “other side,” is that they are rational 
rational in the sense that certain acts would be perceived by all 
participants or observers as being beneficial or detrimental 
either with respect to cost for benefit gained or to survival. And 
admittedly, assumptions of this nature are necessary starting 
points. But they have their weaknesses. 

For example, from a rational standpoint of cost effective¬ 
ness, the procedures used by the scientists and engineers on the 
Manhattan Project to develop the atomic bomb were fantasti¬ 
cally expensive. Not knowing which of two methods would 
work, General Leslie Groves ordered the construction of two 
production facilities—one using the gaseous diffusion method 
and the other using a centrifugal method for producing 
fissionable material. Both methods had only a theoretical 
chance of success, and both procedures were very costly. But 
the stakes were high, and the fear that Germany would develop 
fissionable material first made the risk of wasting millions of 


4 One of the more ambitious undertakings in identifying, coding, and 
analyzing patterns of national behavior for the purpose of predicting 
future behaviors of nations as they interact with other nations is the DON 
(Dimensionality of Nations) project. Employing factor analysis and linear 
algebra, the project attempts to identify major attribute patterns in terms 
that would permit the analysis of cross-national data. The attribute 
patterns include “size,” “economic development,” “political orientation,” 
“density,” “Catholic culture,” “foreign conflict,” and “domestic con¬ 
flict.” The project analyzed over 100 behavioral variables for about 400 
two-nation combinations. The focus of the project is to identify 
regularities in the relationship of attributes and behavioral variables- 
regularities that would serve as the basis for predicting future behavior. 
The research was performed by the Department of Political Science and 
the Social Science Research Institute of the University of Hawaii under a 
grant provided by the National Science Foundation. 
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dollars an acceptable one. In light of the monumental technical 
difficulties that had to be overcome, and in light of the 
tremendous costs in time, manpower, and facilities, it is little 
wonder that the highly rational German scientist Werner 
Heisenberg refused to believe that the Americans had in fact 
detonated successfully three atomic devices. 

Heisenberg’s incredulity about American atomic capabilities 
reflects a host of other assumptions that prevail when behavior 
is anticipated. For example, Heisenberg assumed that because 
the highly touted German physicists could not devise methods 
for producing large amounts of fissionable material, neither 
could any other group of physicists. 

For years American planners assumed that the Japanese 
would never surrender in battle, and, if they were captured, 
would refuse to cooperate with their interrogators. Both 
assumptions initially had a degree of face validity, but subse¬ 
quently proved invalid. 

Success in battle is often a function of risks, and risks are 
not rational in the conventional sense of the word. Assumptions 
are a necessary part of every predictive technique. Assumptions 
serve a pragmatic function in the sense that they provide a basic 
framework on which a predictive model can be constructed. In 
addition, assumptions are necessary for planners to choose an 
optimum course of action. As T. C. Schelling stated, 

In any analysis that leads to a choice of a particular 
strategy or weapon system from among several alterna¬ 
tives, there is typically some assumption about the 
behavior of the enemy. If it is not explicitly stated, it is 
embedded somewhere in the analysis.^ 

But in the realm of anticipating behavior, the unvalidated 
assumption can debase the whole predictive exercise. 


^T. C. Schelling in E. S. Quade, Analysis for Military Decisions 
(Chicago: Rand McNally & Company, 1964), p. 199. 
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Delphi Techniques 

The simplistic solution to the problem of predicting what 
will happen in the year 19— would be to ask the experts. And 
decision makers have been “asking the experts” for centuries 
(with varying degrees of success). 6 On the assumption that the 
collective judgment and wisdom of several experts is better than 
the estimates or predictions of one, a technique for eliciting and 
combining judgments systematically from a group of experts 
has been developed. Appropriately, the technique is referred to 
as the Delphi technique. 

This is how the technique is applied. A series of questions is 
asked of each expert. The experts submit their judgments 
individually. The results of all of the judgments are tabulated 
and these results are sent back to the experts for modification. 
In essence, the experts are asked to reevaluate their original 
estimates in light of the estimates and the rationales for the 
estimates submitted by other experts. The results of the second 
iteration are tabulated, and the new results are again sent back 
to the experts for revision. The process continues until a fairly 
high degree of consensus is reached, or until the experts would 
no longer modify their previous estimates. 

Typically, the technique has been used in technological 
forecasting. But the technique could be used in any type of 
problem-solving situation for which there was no answer, but 
only varying opinion. For example, in an early study conducted 
by RAND, the technique was used to address issues such as 
predictions of scientific breakthroughs, predictions of the 
impact of automation, predictions on progress in space, and 
predictions of weapon systems of the future. 

The technique can be employed in a number of different 
ways. For example, questionnaires can be administered 


6 Prophets, augurers, soothsayers, and haruspices provided this service 
in the past. Delphi technique derives from the Greek oracle who lived in 
Delphi. 
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anonymously so that one expert would not know the identities 
of other experts, and the results of these questionnaires could 
be processed manually. Another procedure would be to have 
the experts interact directly with each other, either in a 
face-to-face meeting or by use of teleconferencing techniques. 
Still another method would be to employ teletype devices 
connected to computers’ memory devices. Although expensive 
and restrictive (in the sense that the expert must be near a 
terminal), this method permits the computers to perform the 
relatively simple statistical operations (e.g., calculating means, 
medians, and quartiles), to store and retrieve earlier responses, 
and to present new data in real time. 

The major advantage of the technique is that it permits 
analysts to obtain an objective consensus of expert judgment. 
Another advantage of the technique is that it makes the 
rationale underlying a specific estimate or prediction explicit 
for everyone. 

The weakness of the technique is that a truly perspicacious 
expert’s judgment might be lost when a consensus that actually 
represents a range of judgments is produced. In face-to-face 
situations, for example, one expert might be swayed more by 
the rhetoric than by the validity of another expert’s argument. 
But the potential for this problem exists whenever humans 
interact, and is not necessarily a weakness of the Delphi 
technique per se. Administering the technique remotely and 
anonymously is one way of reducing the impact of “rhetorical 
intimidation.” 

For additional information on the application of the 
technique, the following sources might be consulted: 

Norman C. Dalkey, The DELPHI Method: An Experimental 
Study of Group Opinion (Santa Monica: Rand Corporation, 
RM-5888-PP, June 1969). 

James R. Bright, ed., Technological Forecasting for Industry 
and Government (New York: Prentice Hall, 1968). 
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Theodore J. Gordon and Olaf Helmer, Report on a Long-Range 
Forecasting Study (Santa Monica: Rand Corporation, Septem¬ 
ber 1964). 

Bernice Brown, T. J. Gordon, and Olaf Helmer, Appendix to 
the Report on a Long-Range Forecasting Study (Santa Monica: 
Rand Corporation, September 1964). 

Murray Turoff, The Design of a Policy Delphi ; Executive Office 
of the President, Office of Emergency Preparedness, Technical 
Memorandum TM-123, February 1970. 

The Delphi technique would be applicable to a range of 
estimative/predictive-type problems encountered in Economic, 
Political, Scientific and Technical, and possibly, Armed Forces 
Intelligence research and analysis. 

Generation of Alternative Futures 

Although similar to the Delphi technique, the generation of 
alternative futures does not require the systematic eliciting of 
experts’ judgments until a consensus is reached. Instead, the 
generation of alternative futures uses the creative powers of a 
single person or of a group of experts working together to 
produce a number of scenarios. 

Basically, the generation of alternative futures involves 
specifying certain assumptions about a people, a country, an 
international situation, or a technological development, and 
then inferring the various outcomes that might result under the 
stated conditions. Alternative futures are usually cast in the 
form of scenarios—narratives which describe conditions, be¬ 
haviors, and acts of people and their impacts on a future 
environment. 

Alternative futures are not predictions in their own right. 
They are simply descriptions of possibilities. Alternative futures 
can be ranked in terms of possible, plausible, and most likely 
outcomes, but they are not predictive in the sense that what is 
forecast must necessarily come to be. 

Herman Kahn developed a number of alternative futures for 
the 1970’s in which he attempted to describe the results of 
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U.S.—Soviet detente, the effects of political and military 
realignments on NATO, and the effects of evolving military 
technology. In developing his alternative futures, Kahn used 
game theory, systems analysis, and cost-benefit ratios to create 
a set of realistic, plausible futures. In a less detailed fashion, 
Morton H. Halperin, in his short book, China and the Bomb , * 7 
also generated a series of alternative futures pertaining to the 
People’s Republic of China’s acquisition of a nuclear capability 
and delivery system. 

Alternative futures are often used in policy formulation. 
Although one cannot necessarily predict which of several 
“futures” will result, one can usually anticipate what the results 
of certain futures will be. Alternative futures, in a more limited 
sense, are also generated when scenarios of war games are 
prepared. Interestingly, one of the purposes for which a game 
may be played is to determine (quasi-empirically) what the 
implications of certain “futures” may be. 

There is no single technique that comprises the method¬ 
ology for generating alternative futures. In certain cases the 
Delphi technique has been used, and in other cases, simple, 
straight-line extrapolations of present conditions into the 
future have been used. Experts familiar with the problems being 
addressed can identify limiting factors and modify the straight- 
line extrapolations accordingly. Like every other technique 
which utilizes judgment, expert knowledge of the subject 
matter is essential if the alternative futures are to have any 
validity. 

For examples of works which incorporate alternative 
futures or discuss problems involved in generating alternative 
futures, Herman Kahn’s paper “Alternative World Futures” 8 


^Morton H. Halperin, China and the Bomb (New York: Frederick A. 

Praeger Publisher, 1965). 

8 Herman Kahn, “Alternative World Futures,” paper Hi-342-B IV, 
(New York: Hudson Institute, April, 1964). 
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and Alvin Toffler’s Future Shock ^ should be examined. 

Extrapolation 

One of the more common methods of forecasting future 
conditions or trends involves projecting into the future the 
trends or conditions that have existed in the past and exist at 
the present. In its simplest form, this “extrapolation” technique 
involves extending a line of a graph to depict the anticipated 
changes in the future on the basis of the information known to 
date (Figure XVIII-1). 



Figure XVIII-1. Simple “Straight Line” Extrapolation 


In the first example of extrapolation (Figure XVIII-2), a 
constant number of items was added each year. In other words, 
N items (y 2 - yi) were added during time T(x 2 - xj), and the 
numerical change, or N/T, was constant. The increase in many 
phenomena through time is constant, so the figure which 
describes those phenomena through time is a straight line. 


9 Alvin Toffler, Future Shock (New York: Random House, 1970). 
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Some types of phenomena are not described well by straight 
lines, however. Population growth is a case in point. Plotting 
population growth is like compounding interest. As the number 
of people in a population increases, so does the base which 
produces subsequent generations increase. 

The number of people in a population which is growing by a 
constant percentage can be illustrated by an exponential curve 
(Figure XVIII-3). The curve can be drawn by hand, or a formula 
can be applied to the known data. The simplest form for the 
equation which describes a constant percentage growth rate is 

y 2 = yjO + r) N 

where y^ equals the current population, r equals the yearly 
percentage increase in population, N equals the number of years 
between the present (xj) and the year for which the population 
projection is being made (X 2 ), and y equals the projected 
population in year X 2 . 

When this formula is employed, r, the yearly percentage 
increase in population, is computed by using the known 
population levels in the past few years. Most of the points 
which represent the known population levels do not lie on the 
curve. When the formula is used, however, the projected 
populations for future years do lie on the curve. This 
characteristic is common to all extrapolation techniques. It is 
not uncommon for percentage rates of change to vary slightly 
so that a projected population may be slightly higher or lower 
than the actual population. 

Seemingly simplistic, extrapolations based on the known 
dynamics of a system are not at all trivial. And although it takes 
little skill simply to extend a line on a graph, it does take 
considerable insight and judgment to know how to alter that 
line. 

Short-range projections are relatively simple to make, and 
simply extending a line of a graph which represents current 
conditions usually suffices for predicting the immediate future. 
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Figure XVIII-2. Extrapolation with a Constant Change 
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But long-range projections—the projections that are usually of 
greater interest to forecasters—are made up of not single 
continuities, but rather a series of continuities (Figure XVIII-4). 
The factors that affect or determine the overall series of 
continuities must be analyzed before a researcher is able to 
determine when, how much, and in what direction that 
extrapolated line on a graph should be bent. 



Figure XVIII-4. Long-Range Projection based on a Series of 

Continuities. (Lines with arrows would be short- 
range projections based on the immediate past 
and current information.) 

Knowledge of plans, knowledge of the ways certain vari¬ 
ables behave characteristically, and knowledge of special condi¬ 
tions provide clues in predicting trends and making projections. 
For example, precise knowledge of a government’s budget for 
the current and forthcoming fiscal years provides some indica¬ 
tions as to the extent that a current program will continue into 
the next fiscal year. And this information, coupled with 
information pertaining to the previous fiscal year, would give a 
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fairly strong indication of the continuity of a trend at least for a 
short-range period. 

In other cases, certain variables may tend to behave in fairly 
predictable modes, and these modes provide some basis for 
projecting and interpreting series. For example, depletions of 
inventories generally herald an increase in production, and the 
introduction of new models usually brings about increased sales. 

Knowledge of special conditions can also be used as a basis 
for prediction. For example, foreknowledge that interest rates 
will increase can be used to anticipate a decrease in new housing 
construction. And a major technological advance in a nuclear 
delivery system can be expected to bring about a new series of 
arms races. 

Extrapolations can also be refined by knowledge of limiting 
conditions. For example, a country could increase its military 
manpower annually but only until, theoretically, every individ¬ 
ual in that country would be in uniform. Clearly, this projection 
should be refined to exclude the aged, the very young, and 
those engaged in critical activities such as government, agricul¬ 
ture, and industry. One could have projected the increase in size 
in U.S. Naval ships, but until the World War II era, a limiting 
factor on the size of a naval ship was the size of the locks on 
the Panama Canal. Limiting factors may be physical constraints, 
cultural constraints, “laws of nature” (e.g., gravity, speed of 
light), or economic constraints. 

Obviously, there are dangers to “blind” extrapolation- 
dangers such as concentrating on short - range trends to the 
exclusion of long - range trends. However, one way of mini¬ 
mizing the errors in extrapolation is to limit the span of future 
projections to the span of time for which historical data exist. 
Another way of minimizing error is to avoid predicting specific 
details. For example, in technological forecasting attempts are 
sometimes made to project technological developments based 
on an analysis of each component of a system. The assumption 
is that the performance limits of the components will, collec¬ 
tively, determine the performance limits of the system or 
device. Historically, however, extrapolations based on this type 
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of analysis have proved too conservative because major inven¬ 
tions (“breakthroughs”) typically changed the very nature and 
basic configuration of the system or device. 10 

Another way of minimizing error is to express forecasts in 
terms of envelope curves which express the upper and lower 
limits of expectations and which are broad enough to accommo¬ 
date normal contingencies and a normal rate of innovation 
(Figure XVIII-5). 



Figure XVIII-5. An Extrapolation with an “Envelope Curve” 
Showing the Upper and Lower Limits of 
Expectations 


10 The advent of the pocket electronic calculators might be cited as a 
case in point. Any forecaster who, forty years ago, would have projected 
the extent to which calculators would be in common use, probably would 
have made a very conservative estimate had he based the estimate on the 
current state of the art. Vacuum tubes, size, and weight of the electronic 
calculating devices of the 1940’s, to say nothing of costs, made these 
devices impractical for general use. But the development of printed 
circuitry and solid state components reduced the size, weight, and costs of 
the devices, and increased the speed and reliability. Solid state components 
were not mere extensions of technology: they were revolutionary 
developments. 
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Extrapolation techniques are often discussed in works 
related to technological forecasting. The references suggested in 
the section dealing with morphological analysis ( p . 325) discuss 
extrapolation techniques as well. Extrapolation techniques 
might be applied to Armed Forces Intelligence, Scientific and 
Technical Intelligence, Sociological and Economic Intelligence 
research and analysis. 

Bayesian Analysis 

Intelligence analysts are often asked to make a quick 
assessment of the likelihood of occurrence of an event. The 
initial assessment must often be made off-the-cuff, on the 
basis of information already known about the problem. 
Bayesian analysis allows the analyst to update his probabilities 
systematically as new information becomes available. 

For example, an analyst might be asked to assess the 
probability that a certain developing nation will produce a 
nuclear device. He formulates two hypotheses. 

H j: The nation will develop a nuclear device 

H 2 : The nation will not develop a nuclear device 

and he assesses the prior probability that Hj will occur as .2 and 
the prior probability that H 2 will occur as .8. Stated anothei 
way, the odds are .2 to .8 (or 1 to 4) that Hj will occur. 


11 Stated still another way, the analyst feels that the likelihood that a 
certain nation will not develop a nuclear device is four times greater than 
the likelihood that a nation will develop a nuclear device. 
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The analyst now seeks new information that might influ¬ 
ence the likelihood of Hj. His analysis differs from probability 
diagram analysis (which will be described later) in that initially 
he does not try to outline all of the major conditions upon 
which H] depends. Instead, he revises the probability of Hj 
every time he receives a new piece of information. 

After making his initial assessment, the researcher might 
note that a cadre of nuclear physicists and engineers trained in 
the Soviet Union has returned to the country in question. The 
analyst decides that this fact (D) influences the probability of 
Hi. He is now ready to apply Bayes’ theorem. 

Bayes’ theorem is usually expressed as: 


p(Hj|d) 


PCDlHp P(Hj) 
P(D) 


( 1 ) 


where 

P(D) = probability of the datum occurring 

P(Hj) = prior probability of hypothesis Hj 

P(DlH 1 ) = probability of the datum occurring given that 
the hypothesis is true 

P(Hj ID) - posterior, or revised, probability that the 
hypothesis is true given that the datum has occurred. 

It is often easier for many analysts to think in terms of odds 
than in probabilities. The odds which assess the relative 
likelihood of occurrence of Hj and H 2 may be expressed by 
repeating Bayes’ theorem for both hypotheses: 


P(Hj | D) 


PCDlHQPCH!) 

P(D) 


( 2 ) 


and 
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P(D|h 2 ) P(H 2 ) 


(3) 


P(HJD) =- 

2 P(D) 

When equation (2) is divided by equation (3), the P(D)’s cancel 
out and the following remains: 

PCHjD) = P(D|H!) P(H t ) (4) 

P(H 2 |D) ~ P(D|H 2 ) P(H 2 ) 

or, using the most common verbal definitions, 

Posterior Odds = Likelihood Ratio x Prior Odds (5) 

Equation (4) can now be used to revise the prior odds to 
posterior odds as a function of the information in the observed 
datum. To go from prior odds to posterior odds one must 

assume first that Hj is true and then assume that H 2 is true. 

The analyst than asks himself how likely it is that the datum 
would be observed under each assumption. If P(DlHi) is 
greater than P(DlH 2 ), the posterior odds are greater than the 
prior odds. If P(DlH 2 ) is greater than P(DI Hj), the prior odds 
are greater than the posterior odds. 12 

In the example of the nation which may or may not be 
developing a nuclear device, P(Hj) was already assessed as .2. 
The analyst estimates that it is three times more likely that i 
large number of people would be educated in nuclear physics 
and related fields if the nation were proceeding toward a 
nuclear capability than if the nation were not working toward 
such a capability. The equation to compute posterior odds for 
this example is: 


12 This discussion is based on C. R. Peterson etal., “Inference from 
Evidence: Bayes’ Theorem,” Handbook for Decision Analysis (Washing¬ 
ton, D.C.: Defense Intelligence School, 1973). 
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( 6 ) 


P(HjlD) 3 x .20 .60 1.00 
P(H 2 |D) ” 1 x .80 ~~ .80 ” 1.33 

Therefore, the odds concerning the nation’s progress toward 
a nuclear capability have increased from 1:4 to 1:1.33. 

The analyst assigns a 2:1 likelihood ratio in favor of the 
purchase of fuel if Hj, nuclear device development, is true. 
Using the posterior odds computed in (6) as the new prior odds, 
he calculates: 

P(Hi|D) 2x 1.00 2.00 1-50 

-i- = - = — = - (7) 

P(H 2 |D) 1x1.33 1.33 1.00 

For the first time during the current analysis, the odds have 
increased so that they are greater than 50:50. It now seems that 
it is more likely than not that the nation will develop or is 
developing a nuclear device. 

This procedure could continue indefinitely. The analysis 
could be used by the researcher as a method to monitor a 
situation. It could also be used to determine when to begin to 
make policies relating to a situation. In the current example, it 
might be decided to try to seek information more directly once 
the odds were 5:1 in favor of Hj. 

Bayesian analysis may be extended to a many-valued case 
when more than two hypotheses are under consideration. The 
following formulas address the many-valued case: 

. P(D I Hj)P(Hj) 

P(Hi ID) =- 1 - - (8) 

P(D) 

where Hj rs each possible hypothesis. All the P(Hj) must sum to 
one. 
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In the two-valued case, the initial formula was converted to 
equation (4) so that the analysis could proceed by the 
assessment of odds rather than probabilities. In a many-valued 
case it is more difficult to keep track of all the likelihood 
ratios. Although the basic mathematical operations are the 
same, the short cut for assessing posterior odds appears 
different in the many-valued case. 

The analyst begins by listing all prior hypotheses and 
associated probabilities (Table XVIII-1, columns 1 and 2). Next 
he assigns likelihoods to P(DlHj), (Table XVIII-1, column 3) 
He multiplies column 1 values by column 2 values which gives 
him the numerator of equation (8), or P(D I Hj)P(Hj). Since the 
sum of all posterior probabilities must be 1, 

2 P(Hj)P(D I Hj) 

n i i= 1 

2 P(Hj | D) - 1 = - (9) 

i=l P(D) 


or 


P(D) = 2 P(Hi)P(DlHi). (10) 

i=l 

Therefore, column (4) is summed to obtain P(D). Finally, each 
P(Hj)P(D |Hi) is divided by P(D) to obtain P(Hjl D), (column 5). 

Bayesian analysis is appropriate for assessing the probability 
of occurrence of an event that might be influenced by many 
other events. The intermediate events should be used as data 
(D) when it seems more helpful to apply Bayes’ theorem than 
to assess probabilities directly. 

For more information on the application of Bayes’ theorem, 
the following works should be consulted: Ward Edwards 
introductory comments to the March 1966 edition of IEEE 


316 


Table XVIII-1. A Three-Valued Bayesian Analysis 


O) 

HYPOTHESIS 

(2) 

PRIOR 

PROBABILITY 

(3) 

LIKELIHOOD 

(4) 

JOINT 

PROBABILITY 

(5) 

POSTERIOR 

PROBABILITY 

Hi 

P(H i ) 

P(D |H;) 

P(Hi) P(DlHi) 

P(H i 10) 

NUCLEAR 

CLUB 

o 2 

. 9 

,18 

,29 

POWER 

PLANTS 

.5 

.7 

o 35 

,56 

NO NUCLEAR 
INTEREST 

,3 

,3 

. 09 

„ 15 




P (0 )= . 6 2 

(v\ AAaiUAi. 

1.00 


V 


Transactions on Human Factors in Electronics^^ . Also the 
article by L. D. Phillips et a/ 14 in the same issue should be of 
interest to researchers considering applications of Bayes’ 
theorem. Edwards, it should be noted, conducted research in 
the application of Bayes’ theorem to intelligence problems. For 
the more serious student of the methodology, Bayesian Sta¬ 
tistics by Donald L. Meyer and Raymond O. Collier, Jr. would 
be appropriate.' ^ 


'-'Ward Edwards, introductory comments in IEEE Transactions on 
Human Factors in Electronics, HFE-7, (March 1966): 3. 

'^L. D. Phillips, W. L. Hays, and Ward Edwards,“Conservatism in 
Complex Porbabilistic Inference, “ IEEE Transactions on Human Factors 
in Electronics, HFE-7, (March 1966): 7-18. 

'^Donald L. Meyer and Raymond O. Collier, Jr., Bayesian Statistics 
(Itasca, Illinois: F. E. Peacock Publishers, Inc., 1970). 
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Figure XVIII-6. Different Branching Diagrams of the Same Phenomenon 





























probability diagram, the fractional probabilities of two 
mutually exclusive branches must sum to 1. 

An expert estimates the probability of Country P dropping 
out of the alliance as .7 before he begins an in-depth analysis. 
He is not basing this probability on any assumptions or 
branches in the probability diagram, so it is an unconditional 
probability estimate. 

On the other hand, when the expert begins to study the 
problem in depth, he decides that the outcome of a future 
election for national assemblies is the major condition upon 
which Country P’s alliance status would depend. Similarly, the 
outcome of the election depends upon the value of Country P’s 
currency, and the value of the currency depends upon whether 
or not Country P experiences a trade deficit this year. Once the 
analyst identifies all the major conditions upon which the status 
of Country P’s military alliance depends, he can build a 
probability diagram (Figure XVIII-7). It is critical that the 
analyst diagram the events in the proper order. For example, if 
the value of currency depends upon the balance of trade, the 
value of currency must branch from the outcomes of the 
balance of trade, not the other way around. 

Numerical probabilities are assigned to each branch, begin¬ 
ning with the first level of branching (trade deficit or no trade 
deficit, in this case). The trade deficit and no trade deficit 
probabilities must sum to 1.0. The expert decides, based on all 
his past experience, to assign a .7 probability to trade deficit, 
and a .3 probability to no trade deficit. In order to assess 
probabilities at the next level of branching, it is assumed, for 
the time being, that a given outcome has occurred. Given that a 
trade deficit has occurred, either there will be a devaluation of 
currency or there will not be a devaluation of currency. 

A path probability can be traced for each unique set of 
conditions and outcomes. In the alliance example, a path 
probability is traced for the uppermost branches (Figure 
XVIII-8). The conditional probability that P will remain in the 
alliance given a Communist majority in the national assembly, 
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Figure XVIII-7. A Probability Branching Tree 
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Figure XVIII-8. Probabilities Assigned to Branches 
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devaluation of currency, and a trade deficit is obtained by 
multiplying all the individual probabilities of these branches. 

P 1 x p 2 x p 3 x p 4 = 7 x - 7 x 8 x • 1 = .0392 

At the last level of branching, all of the outcomes based on 
all paths sum to one. 

One of the more important outcomes of the example 
problem would be to determine the total probability of P 
dropping out of the alliance regardless of the conditions causing 
this outcome. Addition of all path probabilities for the given 
final outcome solves such a problem. In the example (Figure 
XVIII-7), .3528 + .0294 + .0672 + .0378 + .0432 + .0072 + 
.0294+ .0168 = .5838. 

The expert can now compare the computed probability of 
about .6 with his estimated probability of about .7, and he may 
decide to revise his estimate on this basis. 

Sensitivity analysis allows a person to assign a range of 
probabilities rather than a single probability to a 
condition. Final outcomes are computed based on the upper 
and lower limits of the range, and the effect of the range on the 
final outcome may be determined. Sensitivity analysis can 
indicate conditions which should be studied at greater depth. 

Probability diagrams are tools for identifying the conditions 
which lead to certain outcomes. But it must be remembered 
that the assigned probabilities are subjective and are usually 
based upon the opinions of a few people. The final results of a 
probability diagram analysis should be considered only as 
approximations, and the values should be rounded off. 

Probability diagrams, and their application to intelligence 
problems are discussed in detail in C. R. Peterson et al.. 
Handbook for Decision Analysis , cited earlier. 
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Psycho-Historical, Psycho-Linguistic 

Analysis 

Behavior patterns as manifested in speech (oral or written) 
have been used for purposes of analysis (description) and 
prediction. The predictions are based on established associations 
between the words and deeds of a principal actor, a head of 
state for example. In World War II, British propaganda analysts 
established that German propagandists, particularly Joseph 
Goebbels, would not boast of capabilities which Germany did 
not have because the expectations of the German population 
would soar and then plunge precipitously when anticipated 
results did not come about. Conversely, when the German 
propaganda machine did begin to allude to new terror weapons, 
the analysts inferred correctly that a new weapon was nearing 
operational status. (The weapons were the V-l and V-2 
rockets.) 

Numerous other studies have attempted to correlate threat¬ 
ening statements of leaders with subsequent activities that their 
countries carried out. More conventional attempts to establish 
these associations have usually involved interdisciplinary 
teams—teams composed of, say, political scientists, historians, 
sociologists, and anthropologists. In recent years, psychologists 
and psychoanalysts have also participated in these types of 
analyses. Embracing in part sociology and anthropology, 
psychoanalytic theory attempts to establish the psychological as 
well as the cultural determinants of a principal actor’s behavior. 
Linguistics, a branch of anthropology, attempts to define the 
role that language plays in determining a member of a certain 
culture’s perception of the world and consequently, his behav¬ 
ior patterns. 

In intelligence, for the greater part, psycho-linguistic analy¬ 
ses would address contemporary figures. But psycho-linguistic 
analysis (or psycho-historical analysis) has been performed on 
historical figures as well. For example, Jean Leclerq, presently 
Abbaye of Clervaux, made a psycho-linguistic analysis of the 
writings of William of Saint-Thierry, a twelfth century abbot. 
William’s writings were compared to writings of his contempor¬ 
aries, his accounts of events were compared to other historical 
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and economic data, and the consistency of his perceptions 
throughout his writings was analyzed. On the basis of this 
analysis, a psycho-analytic “profile” was made which revealed, 
for example, that William tended to project pessimism to the 
subjects he wrote about, notably, St. Bernard. 

Similar types of analyses have been performed on public 
pronouncements by contemporary heads of state, and tentative 
hypotheses pertaining to the relationship between words and 
deeds have been established. Given a notably vociferous head of 
state, Fidel Castro, for example, numerous opportunities for 
testing the hypotheses may exist. 

Psycho-linguistic (as well as many psychoanalytic) analyses 
may be very tenous. For example, the analysts are usually 
addressing figures of a culture different from theirs. As anyone 
who has done any cross-cultural analysis knows, interpretation 
of behavior such as speech, which carries connotatiye as well as 
denotative meaning, is subject to numerous failings and error. In 
many cases the validity of the basic psychoanalytic theory itself 
is in question. 

Psycho-linguistic, and psycho-historical analyses are not 
without their detractors. Kenneth M. Colby, a psychoanalyst 
himself, says: 

Psychoanalytic studies of Moses, Leonardo, Poe, and 
Shakespeare are fascinating and entertaining. But they 
are not science; they belong to art, literature, or 
history .... Science is interested in useful generaliza¬ 
tions about a class. An individual biography may 
illustrate some generalizations arrived at through other 
evidence .... These ideographic, rather than nomothe¬ 
tic, efforts may result in literature, but not in science. 
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When art and science accost the same questions they 
have quite different criteria for acceptable answers. ^ 

Although almost any careful researcher can establish rela¬ 
tionships between words and deeds (if they exist), attempting 
to postulate reasons for the behavior (interpret behavior) 
requires highly specialized knowledge and skill. As such, 
psychoanalytic techniques are better left to the experts. 

Psycho-linguistic analyses would be appropriate for Political 
Intelligence research and analysis, as well as for Sociological and 
Biographic Intelligence. Examples of intelligence analyses based 
upon associations between public announcements and subse¬ 
quent acts are described in Alexander L. George’s Propaganda 
Analysis 

Morphological Analysis 

Morphological analysis is a technique that was originally 
devised for technological forecasting. Since its introduction in 
the 1940’s, the technique has been applied to exploratory 
studies of future geo-political environments and situations as 
well. 

At its most general level, morphological analysis is a method 
of multi-dimensional classification. 18 Fritz Zwicky, the astro¬ 
physicist and engineer who pioneered the use of morphological 
analysis, classified the totality of a ll jet engines operating in a 

^Kenneth Mark Colby, An Introduction to Psychoanalytic Research 
(New York: Basic Books, Inc., 1960), p. 74. That different types of 
research (description and prediction, for example) have different criteria 
for acceptance was a point stressed in Chapter VII, “Types of Inquiry and 
the Nature of Proof.” 

^Alexander L. George, Propaganda Analysis: A Study of Inferences 
Made from Nazi Propaganda (Evanston, Illinois: Row Peterson and 
Company, 1959). 

^From Robert U. Ayres, Technological Forecasting and Long-Range 
Planning (New York: McGraw-Hill Book Company, 1969), p. 73. 
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Table XVIII-2. Zwicky’s Classification of Jet Engines^ 


P , 1.2 

INTRINSIC OR EXTRINSIC CHEMICALLY 

ACTIVE MASS 

(2) 

P 2 1 , 2 

INTERNAL OR EXTERNAL THRUST GENERATION 

(2) 

P 3 1,2,3 

INTRINSIC, EXTRINSIC, OR ZERO THRUST 
AUGMENTATION 

(3) 

p 4 1>2 

INTERNAL OR EXTERNAL THRUST AUGMENTAT 1 ON 

(2) 

Pgf.2 

POSITIVE OR NEGATIVE JET 

(2) 

Pb 1 . 4 

POSSIBLE THERMAL CYCLES (ADIABATIC, 
ISOTHERMAL,ETC,) 

(4) 

p 7 1.4 

MEDIUM (VACUUM, AIR, WATER, EARTH) 

(4) 

Pg 1 ' ■ • • . 4 

MOTION (TRANSLATORY,ROTA TORY,OSCILLA¬ 
TORY. NONE) 

(4) 


1 ^Ibid. The canons of classification discussed in Chapter XV are most 
relevant to morphological analysis. 
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pure medium (e.g., vacuum, air, water, earth), which contained 
only simple elements and which were activated by chemical 
energy. His classification looked like Table XVIII-2. (The 
number of variations of each class is shown in parenthesis.) 

Excepting the instances in which certain classes were 
self-contradictory (e.g., internal or external thrust generation 
would not apply in conditions of zero thrust augmentation), 
25,344 possible jet engine combinations were generated. By 
examining various combinations, Zwicky was able to propose 
radically new inventions-inventions which were later developed 
successfully. 

Zwicky states four rules for performing morphological 
analysis: 

1) The problem to be addressed must be stated with great 
precision. (This is a fundamental rule for all types of analyses or 
problem-solving activities, including intelligence research.) 

2) The parameters or characteristics of that which is being 
analyzed must be identified. (This identification should be 
complete—a requirement that is more easily stated than 
accomplished.) 

3) Each parameter or characteristic must be subdivided 
into specific cases or “states” sometimes called extensional 
characteristics. (This step may be difficult to perform when 
continuous phenomena are involved. For example, in classifying 
objects on the basis of speed, the division between supersonic 
and hypersonic is a tenuous one.) 

4) The implications of the various combinations of exten¬ 
sional characteristics must be analyzed for possibility, plausi 
bility, feasibility, practicality, and so on, depending upon the 
nature of the phenomenon analyzed, e.g., a technological 
process, a device, or a military or political condition. 

2dxhis incident is similar to the one in which Dmitri Mendeleev 
predicted the properties of elements which had not yet been discovered 
(Chapter XV). Tire procedures involved in constructing the periodic table 
were nearly identical to the procedures of morphological analysis. 
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In the example below (Table XVIII-3), researchers were 
concerned about possible new technological developments of 
missiles. On the basis of combinations of extensional and 
intensional characteristics of missiles which exist today, a 
matrix showing actual and possible combinations of missile 
characteristics could be generated. 

In a morphological analysis of possible future conflicts, the 
intensional and extensional parameters might look like Table 
XVIII-4. Matrices of the type shown here have been used to 
generate conflict situations for gaming exercises. 

Morphological analysis shows only the possibilities that may 
result given the combination of various elements: it says 
nothing about the likelihood that certain combinations would 
ever be realized. Despite this, the technique is a useful one for 
examining possibilities or for suggesting alternatives to what 
exists at the present time. The potential value of the output of 
the matrices is a function of the quality of the extensional and 
intensional analysis. Again, the thoroughness and creativity of 
the researchers play a critical role. 

For more information on morphological analysis, the 
following works are useful: Morphology of Propulsive Power, 
(Pasadena, California: Society for Morphological Research, 
1962); Robert U. Ayres, Technological Forecasting and Long- 
Range Planning , referred to earlier; and Hans Blohm and Karl 
Steinbuch, Technological Forecasting in Practice (Lexington, 
Massachusetts: D. C. Heath & Co., 1973). Additional materials 
on long-range forecasting can be found in Long Range Fore¬ 
casting and Planning (U.S. Air Force Office of Aerospace 
Research Symposium, August 1966). 

Morphological analysis would be an appropriate methodol¬ 
ogy for Scientific and Technical Intelligence, and, to a lesser 
degree, to Economic, and Political Intelligence research and 
analysis. 
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Table XVIII-3. Intensional and Extensional Characteristics of Missiles 


INTENSIONAL 

CHARACTERISTICS 

EXTENSIONAL CHARACTERISTICS 

Pj 1 ’ 2 ENERGY SOURCE 

LIQUID PROPELLANT- SOLID PROPELLANT 

p 2 '’ 2 ' 3 ' 4 SPEED 

SUBSONIC - SONIC - SUPERSONIC - HYPERSONIC 

P 3 1 ’ 2 ’ 3 RANGE 

SHORT RANGE - MEDIUM RANGE - LONG RANGE 

p 4 1 ’ 2 • 3 '^ GUIDANCE 

COMMAND GUIDANCE - DEAD RECKONING - POSITION FIXING-HOMING 

p 5 1>2 WARHEAD 

SINGLE - MULTIPLE 


Table XVIII-4. Intensional and Extensional Parameters of a Conflict 
Situation (After Ayres)^! 


INTENSIONAL CHARACTERISTIC 

EXTENSIONAL CHARACTERISTIC 

Pj 1,2.n! REGIONS 

EUROPE, MIDDLE EAST, ETC 

p 2 1,2.n 2 LEVEL OF CONFLICT 

SUBCRISIS MANEUVERS,INTENSE CRIS1S,INSURGENCY WAR, 
NUCLEAR WAR, ETC. 

p 1,2.n 3 REASON FOR UNITED 

STATES INVOLVEMENT 

TREATY PARTNER OF ONE BOTH SIDES. TRADITIONAL 

FRIEND OF ONE BOTH PARTIES, IDEOLOGICAL COMMIT¬ 
MENT TO ONE SIDE, UNITED STATES MILITARY BASES 
THREATENED,RAW MATERIAL SUPPLY THREATENED, ETC. 

p 4 1,2.n 4 STRATEGIC OBJECTIVE 

ENGINEER A COUP D'ETAT, STALEMATE,ENCOURAGE NEGO¬ 

TIATION TO SOLVE PROBLEMS, PUNISH ONE SIDE, 
MILITARY VICTORY... 

p 5 1. 2.n 5 TYPE OF INTERVENTION 

DIPLOMATIC.TAKE TO UNITED NATIONS.TRADE BOYCOTT. 
TRADE EMBARGO. BLOCKADE,SEND ADVISORS. SUPPLY 
WEAPONS, AIR SEA INTERDICTION, GROUND FORCES, 
TACTICAL NUCLEAR WEAPONS, ETC. 


21 


Ibid., p. 86. 
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Table XVTII-5. Methodologies and Components of Strategic 
Intelligence 


METHODOLOGY 

COMPONENT OF STRATEGIC INTELLIGENCE 

nwosrnnrwrr 

METHODLOGY IS 
NORMALLY USED 

BIOGRAPHIC 

ECONOMIC 

SOCIOLOGICAL 

TRANSPORTATION 
& TELECOMM. 

MILITARY 

GEOGGRAPHIC 

ARMED FORCES 

POLITICAL 

SCIENTIFIC & 

TECHNICAL 


DESCRIPTIVE 

PREDICTIVE 

REFERENCED 

(CHAPTER) 

ANALOGY 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

XVII 

GAME THEORY 


X 




X 

X 



X 

X 

XVI1 

WAR GAMING 


X 




X 

X 



X 

X 

XVI 1 

LINK ANALYSIS 

X 


X 

X 


X 

X 



X 

X 

XVII 

PSYCHO HISTORICAL 
PSYCHO-LINGUISTIC 

X 






X 



X 


XVIII 

SOCIOMETRY 

X 


X 



X 

X 



X 


XVI 1 

CODING & 

CLASSIFICATION 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

XV 

LINEAR PROGRAMMING 


X 


X 


X 


X 


X 


XVI 1 

STATISTICAL ANALYSIS 
(VARIED) 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

XVI 

DELPHI TECHNIQUE 


X 





X 

X 



X 

XVIII 

GENERATION OF ALTER 

NATIVE FUTURES 


X 

X 



X 

X 

X 



X 

XVIII 

MODELS AND MODEL 
BUILDING 


X 

X 

X 

X 

X 

X 

X 


X 

X 

XVIII 
VI 1 

MORPHOLOGICAL ANALYSIS 




X 




X 



X 

XVIII 

CYBERNETIC MODELS 

SYSTEM DYNAMICS 


X 

X 

X 


X 

X 

X 


X 

X 

XVIII 

EXTRAPOLATION 


X 

X 

X 




X 



X 

XVIII 

BAYESIAN ANALYSIS 


X 


X 


X 

X 

X 



X 

XVIII 

REGRESSION & 

CORRELATION 


X 

X 

X 



X 

X 


X 

X 

XVII 

PROBABILITY DIAGRAMS 


X 


X 



X 

X 



X 

XVIII 

GRAPHIC DEPICTIONS 


X 

X 

X 

X 

X 

X 

X 


X 

X 

XVII 

CONTENT ANALYSIS 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

XV 

XI1 1 
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Models and Model Building 


Models, as mentioned earlier, are abstractions or representa¬ 
tions of reality. Stated another way, models are gross simplifica¬ 
tions of reality. Only those critical portions of reality that are 
essential to decision making (or to any other purpose for which 
the modeling is performed) are represented in models. For 
example, in a model for calculating a missile system’s cost, 
reliability of the missile would have to be represented by the 
model since it is a critical characteristic of the system; but color 
and shape of the missile would be inconsequential and would 
not be taken into account by the model. 

Models, as discussed earlier, can be used to represent 
objects, processes, or functions. Very often models of objects 
take the form of a physical mockup. Physical objects can also 
be modeled symbolically by expressing the object in terms of its 
dimensions and weight. Processes can be modeled statically by 
portraying the sequence of steps or events involved in that 
process. Flow charts, for instance, are static models of 
processes. Processes can also be modeled dynamically. Simula¬ 
tions are dynamic models of processes. Models are sometimes 
theoretical constructs of processes and organizations, for 
example, Graham Allison’s behavioral models of decision 
making under three different conditions of government. 22 

Although models are sometimes expressed in physical form, 
and often in verbal form, for the greater part, models are 
expressed mathematically. The advantage of mathematical 
models is that once they are constructed and validated, any 
number of different coefficients can be substituted in an 
equation. Hence, one basic model permits the researcher to 
determine any number of outcomes, given any changes of values 
provided to the model. For the greater part, large war games are 


22 Graham T. Allison, Essence of Decision: Explaining ’ the Cuban 
Missile Crisis (Boston: Little, Brown &Co., 1971). 
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composed of numerous mathematical models that were de¬ 
signed to address specific sub-elements of a situation; e.g., an 
economic model, a conflict model, a political model, and so on. 

An example of a portion of a model—a mathematical model 
that was developed to compare missile systems’ costs—is shown 
below: 


C = C^m - RCjyjNm - Cj^( 1 - R)m 

In this equation, C is the output of the model, i.e., the dollar 
cost of a system which would be capable of destroying a 
specified number of enemy targets; and 

Cm’ cos * coe ffi c i en ts (i.e., the cost per missile 
fired and the cost of maintaining a missile 
in a ready state), 

R = ground reliability, 

m = the number of missiles ready for firing on 
each salvo. 

N = the number of salvos in the campaign.23 

In this specific example, inputs to portions of the equation 
above would be outputs of other equations which were also 
models. One advantage of models is that they sometimes permit 
decisions to be made even though actual values for all elements 
may not be available. 

Since models and games are so closely related, it is not 
surprising to find that the steps in developing models are similar 
to the steps involved in developing games. R. D. Specht 
identified four steps in model building: 1) identifying the 
factors that are relevant to the question being addressed; 2) 
selecting the quantifiable factors—factors which can be 


22 The example appears in R. D. Specht, “The Why and How of Model 
Building,” Analysis for Military Decisions , pp. 66-67. 
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expressed numerically; 3) aggregating (“lumping together”) the 
relevant, quantifiable factors in order to reduce the complexity 
of the problem; and 4) spelling out in quantitative terms the 
relationships between and among elements. 24 In instances in 
which a real world counterpart to a model exists, a researcher 
would employ a fifth step: he would attempt to validate his 
model by comparing the model’s output with the output (or 
characteristics) of the real world system. 

Employment of the model for calculating the cost of a 
missile system described above would require only a pocket 
calculator, paper and pencil. But obviously the most complex 
models would require the use of a computer. When computers 
are used and when programs are not available for performing 
certain functions, then specific programs must be developed. 
Although this step does not add to the complexity of the 
model, it does add to the cost of implementing the model. 

Practically every academic discipline has an array of models, 
and models are used extensively in intelligence analysis and 
research. For example, there are models of transportation 
systems, models for predicting grain yield, models of missile 
systems, models of various types of conflict, e.g., ASW, EW, air 
battles, and so on. Before attempting to construct a model, the 
researcher is advised to determine what models already exist. 
Model building and model implementation are time-consuming 
and expensive tasks, particularly when long runs on a computer 
are required. 

The library of works on modeling is voluminous. For 
examples of areas in which modeling has been performed, the 
following works might be examined: Investigation of Model 
Techniques: Final Report , DA Project 3H98-00-8000 (a some¬ 
what dated but comprehensive work that covers a variety of 


24 1 bid., p. 68. 
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technical applications of models) 25 ; Brian J. L. Berry and 
Duane F. Marble, Spatial Analysis: A Reader in Statistical 
Geography , 26 and Ronald Abler, John S. Adams, and Peter 
Gould, Spatial Organization: The Geographer’s View of the 
World, 27 (both works dealing with models applied to problems 
in geography but the analogs would be applicable to intelligence 
problems as well); Elwood S. Buffa, Models for Production and 
Operations Management , * 2 ^ and Peter Langhuffs Models , 
Measurement , and Marketing 29 discuss models appropriate for 
Economic Intelligence. James S. Coleman’s Introduction to 
Mathematical Sociology 30 discusses numerous techniques that 
would be applicable to a range of Sociological Intelligence 
analysis and research problems. Other works of possible interest 
include James M. Beshers, Computer Methods in the Analysis of 
Large-Scale Social Systems and Demitris N. Chorafas, 
Systems and Simulations . 22 


25 U.S. Army Electronic Proving Ground, Investigation of Model 
Techniques (Fort Huachuca, Arizona: 1961), AD 275 549. 

2 ^Brian J. L. Berry and Duane F. Marble, Spatial Analysis: A Reader 
in Statistical Geography (Englewood Cliffs, New Jersey: Prentice-Hall, 
Inc., 1968). 

22 Ronald Abler, John S. Adams, and Peter Gould, Spatial Organiza¬ 
tion: The Geographer’s View of the World (Englewood Cliffs, New Jersey: 
Prentice-Hall, Inc., 1971). 

2 ^Elwood S. Buffa, Models for Production and Operations (New 
York: John Wiley & Sons, Inc., 1963). 

2 ^Peter Langhuff, ed. Models, Measurement and Marketing (Engle¬ 
wood Cliffs, New Jersey: Prentice-Hall, Inc., 1965). 

^Coleman, Introduction to Mathematical Sociology (New York: The 
Free Press, 1964). 

2 * James M. Beshers, Computer Methods in the Analysis of 
Large-Scale Social Systems (Cambridge, Massachusetts: Joint Center for 
Urban Studies, 1965). 

22 Dimitris N. Chorafas, Systems and Simulations (New York: 
Academic Press, 1965). 
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Cybernetic Models and System 

Dynamics as Aids in 
Forecasting 

In instances where elements that constitute a system can be 
defined and where the functions the system performs are 
understood (and, ideally, can be expressed in quantitative 
terms), cybernetic (feedback) models can be used as a basis for 
forecasting. Very briefly, a system is an interacting organization 
of people or objects that is united by some common purpose, 
objective, mission, or function. 

Models used to represent systems must account for certain 
system characteristics; for example, they must define the 
boundary of the system. A system boundary specifies the 
components of the system which will be analyzed. The 
boundary includes all of the elements within the system that 
define or determine the system’s behavior. 

Models must also account for the control mechanism or the 
feedback of the system. Feedback mechanisms are means by 
which the functions of a system are regulated. In a closed 
system, the past behavior of a system, or feedback , is used to 
control its present and future behavior. In an open system, 
there is no feedback. Outputs result from inputs, but inputs are 
not affected by outputs in an open system. In simple systems 
there may only be one feedback loop. In highly complex 
systems, there may be numerous feedback loops. 

Feedback loops are either positive or negative. A positive 
feedback loop is one in which an activity or function continues 
and, in effect, gains momentum as it progresses. An example of 
a positive feedback phenomenon in an ecological system would 
be a geometric increase in the population of a species when its 
predators had been eliminated. A negative feedback loop, on 
the other hand, is one that constantly limits a function. A 
thermostat that turns off a furnace when a certain temperature 
level has been reached is an example of a negative feedback 
loop. In an ecological system, a finite food supply that limits 
the size of a grazing herd would be an example of negative 
feedback. 
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Figure XVIII-9. An Automated Supply System Exhibiting a Negative 
Feedback Loop 




































behavior. (An example would be attacking an incipient problem 
and taking appropriate action before the need became real). 33 

These observations, although theoretical, appear as relevant 
to hardware analogs as they do to real world command and 
control systems. For example, systems dynamics models could 
be constructed to describe the U.S.’s behavior in two similar 
incidents, North Korea’s seizure of the U.S.S. Pueblo and the 
Khmer Rouge’s seizure of the Mayaquez. 

In both instances, the objective-seeking behavior related to 
the recovery of a ship and the crew. In the case of the Pueblo, 
there was a notable lag in the response of the system, and, as 
Deutsch pointed out, the probability of attaining an objective 
(i.e., recovering the ship) lessened as the lag increased. In the 
case of the Mayaquez , the gain, i.e., the ratio of the intensity of 
the system’s reaction and the information (the facts of the 
seizure) seemed appropriate to the situation. Hostilities were 
confined essentially to the area in which the ship and its crew 
were detained. Whether or not the lead in the system’s reaction 
was appropriate cannot yet be determined. 

System dynamics modeling requires the same sequence of 
steps required for gaming and simulations described earlier: 1) 
defining the problem; 2) defining the relevant factors and the 
cause-and-effect relationships; 3) expressing the factors and 
relationships in a symbolic form that can be manipulated (e.g., a 
mathematical model); 4) obtaining values for the variables 
either empirically or from experts’judgments; and 5) comparing 
outputs of the model with real world counterparts and revising 
the model accordingly. 

Although the steps in dynamic modeling are simply stated, 
their implementation may be considerably more difficult. It is 


33 H. M. Dathe, “Cybernetic Models as Aids in Normative Fore¬ 
casting,” in Blohm and Steinbuch, Technological Forecasting , p. 42. 
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fairly easy to depict closed hardware systems analogs such as 
industrial control and regulatory systems or certain self¬ 
regulating ecological systems, but defining what constitutes the 
boundary of an economic or social system is another matter. 

The predictive value of dynamic system modeling, like all 
models* depends upon the validity of the model and the 
correctness of the values that are manipulated in the model. In 
addition to the predictive value of system dynamics models, 
however, are the insights that modelers gain of the system that 
is being analyzed. The methodology of system dynamics 
modeling is itself a dynamic and on-going process of refinement 
which starts at a top level of generality and progresses 
downward through levels of greater detail (and complexity) as 
data become available. Consequently, the models can be refined 
and revised until their outputs correlate with the real world 
counterparts. 

System dynamics modeling is one of the many techniques 
that falls under the rubric of systems analysis. Its uniqueness 
lies in the emphasis that is paid to the regulatory and control 
mechanisms in the systems. 

For examples of system dynamics modeling applied to 
social problems, the works of Jay W. Forrester should be 
examined; for example, Industrial Dynamics , 34 Urban Dynam¬ 
ics^ 5 and World Dynamics.^6 Most works about systems 


3 ^Jay W. Forrester, Industrial Dynamics (Cambridge, Massachusetts: 
MIT Press, 1961). 

33 -, Urban Dynamics (Cambridge, Massachusetts: MIT 

Press, 1969). 

3 ^-——, World Dynamics (Cambridge, Massachusetts: Wright- 

Alien Press, 1971). An explication of World Dynamics can also be found in 
Dennis Meadows and Donella H. Meadows, The Limits to Growth (New 
York: Potomac Associates, 1972). (Paperback.) 
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analysis are also relevant to system dynamics modeling since the 
same basic techniques are used. One of the more readable books 
on the mechanics and philosophy of cybernetics—the basis of 
feedback and control of systems—is Norbert Wiener, Human 
Use of Human Beings . 37 System dynamics modeling would be a 
methodology most applicable to Political, Economic, and 
Sociological Intelligence research and analysis. 

Summary 

D>The quality of any prediction is a function of the nature of 
the phenomenon about which the prediction is made as well as 
the underlying assumptions about the phenomenon. 

>The quality of a prediction is also a function of the validity 
of the model as well as of the validity of the assumptions 
reflected in the model. 

t> Some so-called predictive techniques provide the researcher 
with insights into system or organizational behaviors that may 
be more valuable than the predicted outcome that the tech¬ 
nique provides. 

> Listed below are various methodologies described in the last 
two chapters and elsewhere in the text. The components of 
strategic intelligence in which the methodologies are or can be 
employed are indicated; however, no attempt has been made to 
indicate the degree or extent to which a methodology might be 
employed within any component. Clearly, some methdologies 
are much more relevant to one component than to another. The 
predictive or descriptive uses of the methodologies are indi¬ 
cated, and the chapters in which the methodologies are 
described are shown in Table XVIII-5. 


•^Norbert Wiener, Human Use of Human Beings: Cybernetics and 
Society (New York: Avon Books, 1967). (Paperback.) 
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CHAPTER XIX 
PREPARING THE REPORT 


We should tailor our product for top policymakers in the 
government and the military services. These are busy 
men. Our prose should be lean and spare , not verbose 
and discursive. It should not include unnecessary detail 
or gratuitous information. It should be as clear and 
understandable to the lay reader as to the fellow 
professional. It should be written , in fact , for the No. 1 
Lay Reader in Washington. 

From the Directorate for Estimates 
Organization and Operations Manual 

This chapter discusses the last of the four phases of a 
research project, report preparation. In well-planned and exe¬ 
cuted research projects, report preparation begins as soon as 
data become available for analysis. In fact, report preparation 
can begin even in the data collection phase as bibliographies are 
compiled. As was the case in every other phase of the research 
project, planning is essential to ensure that enough time remains 
to prepare the report properly and to ensure that assistance is 
available when it is needed—assistance in the form of typists, 
technical art production specialists, editors, and reviewers. 
Invariably, report preparation requires more time than is 
anticipated. Unfortunately, by the time the researcher has 
reached the report preparation phase he typically has little 
flexibility in time and costs. Again, this makes prior planning all 
the more critical. 

Report preparation connotes writing, and writing consumes 
a large portion of the report preparation phase. But checking 
footnotes, compiling bibliographies, reviewing, revising, and 
proofreading may require nearly as much time as the writing 
requires. Failing to take these tasks into account during the 
planning phase invariably results in frenetic activity in the days 
and hours prior to the time the report must be submitted. 
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This chapter discusses the mechanics of preparing the 
report, considers canons of formal, expository writing, and 
describes basic tools that can assist the writer. 

Format 

The format of a report is generally specified for research 
projects that are assigned. For example, the Organization and 
Operations Manual of the Directorate for Estimates (DIA) 
describes the formats for Defense Intelligence Estimates (DIE), 
the Special Defense Intelligence Estimates (SDIE), the Defense 
Intelligence Estimates Memorandum (DIEM), and the Defense 
Intelligence Estimates Staff Memorandum (DIESM). 1 Similarly, 
nearly every service specifies a format for staff studies as well as 
for specific types of intelligence studies. A word of caution: 
format specifications change periodically, so it is important that 
the latest documents be used as guides. 

In self-initiated research projects, the researcher has more 
flexibility in selecting a format most appropriate to the research 
he is conducting. If the report will be submitted in fulfillment 
of an academic requirement, then policies of the institution 
must be followed. If the writer intends to publish his report in a 
professional journal, then the format required by the journal 
must be adhered to. Rarely are academic theses suitable for 
publication in professional journals without major revision and 
condensation. Theses invariably read like theses and even major 
publishing houses shun them in the form in which they were 
prepared originally. Supervisors or faculty advisers can recom¬ 
mend appropriate formats, and ultimately these people will pass 
judgment on the product. 

Assigned research projects and self-initiated projects often 
serve different purposes, and it follows that formats may differ 


* Defense Intelligence Agency, Directorate for Estimates, Organization 
and Operations Manual , August 30, 1974. In addition, this document 
describes the steps in the production of various reports and contains a 
short section on style. 
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considerably. For example, research projects are assigned in 
order to obtain answers to specific questions. The competency 
of the researcher is taken for granted when the assignment is 
made. The product will be judged on its completeness and 
accuracy, of course, but the researcher will be evaluated only 
indirectly, if at all. 

With self-initiated research projects, and particularly with 
research projects conducted as a part of an academic require¬ 
ment, the research report is used as a means for evaluating the 
technical competency of the researcher. Consequently, content 
that would rarely appear in an assigned report may be very 
important in a self-initiated project. A review of the literature 
would be a case in point. In an assigned research project, the 
reviewers of the report will assume that all relevant and 
pertinent materials had been reviewed; thus large sections of a 
report devoted to this review would rarely be required. In 
academic research, however, advisers want demonstrable proof 
that the researcher had examined the basic works in his area and 
that his conclusions were based on reliable source data. 
Therefore, reviews of the literature might constitute an impor¬ 
tant part of the research product. 

Another big difference between formats of assigned intelli¬ 
gence research projects and formats of self-initiated projects is 
the manner in which the findings are presented. In assigned 
projects, reports generally start with a statement of the 
problem, followed very closely by the conclusions or recom¬ 
mendations, and end with the justification. In most scholarly 
reports, on the other hand, the report first identifies the 
problem, then discusses the factors bearing on the issue, and 
ends with conclusions. 

For the greater part, most research reports prepared as part 
of an academic requirement comprise three sections: 1) a 
preliminary section containing the title-page, approval sheet, 
preface and acknowledgements, a table of contents, and list of 
tables and figures; 2) the text of the report, including sections 
devoted to a statement of the problem, review of literature, 
findings, and conclusions; and 3) the reference materials in the 
form of bibliographies and appendices (if any). 
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A standard work in preparing theses and reports is William 
Giles Campbell’s Form and Style in Thesis Writing .2 In addition 
to sections on footnotes and bibliographic entries, the manual 
also contains sections on forms and format. Especially helpful 
are the numerous specimen forms. Another work that might be 
consulted is George Shelton Hubbell’s Writing Term Papers and 
Reports . 5 This latter work is more comprehensive in the sense 
that it discusses steps in collecting and evaluating information as 
well as in preparing the report. However, it lacks the detail of 
Campbell’s manual. 

The writer who intends to submit his work for general 
publication by other than U.S. Government agencies should 
consult A Manual of Styled This is considered to be the basic 
work relating to typographical and other rules for authors, 
printers, and publishers. 

If a particular organization does not have its own style 
manual and if a report is to be published as an official U.S. 
Government document, then the U.S. Government Style Man¬ 
ual should be followed. 5 

A word of caution: manuals differ in subtle ways and later 
editions of the same manual may have important changes. 


^William Giles Campbell, Form and Style in Thesis Writing (Boston: 
Houghton Mifflin Company, 1954). 

^George Shelton Hubbell, Writing Term Papers and Reports (New 
York: Barnes & Noble, Inc., 1962). 

^The University of Chicago Press, A Manual of Style (Chicago: The 
University of Chicago Press, 1969). 

^United States Government Printing Office, U.S. Government Style 
Manual (Washington, D.C.: Government Printing Office). 
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Regardless of the manual used as a guide for form and format, 
one manual should be used consistently and exclusively .6 

Mechanics of Report Preparation 

It would be presumptuous to propose one method for 
preparing the text of a report. Some writers have elaborate 
“warm-up” rituals involving sharpening dozens of pencils of a 
specified hardness, gathering all necessary tools and materials, 
and reviewing what was written previously. Other writers use 
what is available and commence with no ritual. Some writers 
compose in longhand on lined tablets, but others compose at a 
typewriter. Some writers prepare different drafts on different 
colored paper, whereas other writers prepare all drafts on plain 
white bond paper. Some notable writers write standing up 
(presumably at schoolmasters desks), but more writers write 
sitting down. The list of personal quirks in writing is a long one, 
but writers invariably develop certain procedures which they 
find most effective. 

Comments below relate to procedures which have been tried 
and tested. They are offered as suggestions, not as hard and fast 
rules. 

Prepare draft copy for ease 
in editing 

All draft content should be prepared so that space is 
available for editorial changes and comments. This means wide 
margins and triple-spaced lines. Without this space, corrections 


^For example, the “correct” bibliographic form for a book with more 
than one author, according to Campbell’s Form and Style in Thesis 
Writing , is to invert the order of the first author’s name and then list the 
other authors’ names in the normal fashion of first name, middle initial, 
and last name. Turabian, on the other hand, in A Manual for Writers (the 
manual used in preparing this text), uses the inverted form for all of the 
authors’ names. Kate L. Turabian, A Manual for Writers of Term Papers , 
Theses , and Dissertations (Chicago: The University of Chicago Press, 
1973). 
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are nearly impossible to make, and subsequent drafts will 
contain the same errors as the initial draft. Of all of the 
expenses incurred by report preparation, paper is the least 
consequential. 

Cut and paste 

Invariably, draft content must be rearranged for better 
continuity and emphasis. Rather than retyping entire pages, 
much time and effort can be saved by cutting and pasting. 
Rubber cement dries quickly and cleans up easily. It is 
indispensable for preparing draft content, illustrations, tables, 
and charts. 

Prepare footnotes and 
bibliographic entries as 
writing progresses 

These references should be prepared conscientiously as data 
are collected and copy is drafted. The researcher who attempts 
to reconstruct or to add notes at a later time invites trouble. 
Sources may no longer be available, specific items may be 
forgotten, and the notes may accumulate much faster than the 
writer suspected. 

Have materials available 

It is frustrating to begin to write and then to discover that 
needed materials are not within an arm’s reach. The “warm-up” 
time in writing is notoriously wasteful. Ensuring that the 
materials that will be needed are present helps reduce this 
inefficiency. 

Use outlines appropriately 

It is inconceivable that a research report could be prepared 
without some kind of an outline. In fact, very often an outline 
is submitted at the point when the research project is assigned 
or approved. But the outline is a guide, not a mold, and as 
content is prepared, the writer may find that the original 
outline is no longer adequate for his purposes. Consequently, 
outlines should be adapted to the requirements of the writer 
and modified when necessary. There are limits to these 
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modifications, of course, and major changes would require 
approval from either the supervisor or the faculty adviser. 
Outlines are often used as “delaying tactics” by those who 
procrastinate. One can always claim that the outline needs more 
detail, and one can arrange and rearrange topics interminably. 
But this tactic does not get the report written. When deadlines 
approach it is imperative that the outline be put to rest and that 
the writer gets on with his major task, preparing the content. 

Schedule work sessions 

Starting up and winding down are, for the greater part, 
time-wasting activities. Efforts should be made to keep this 
wasted time to a minimum. One method is to schedule 
relatively large blocks of time for writing. An hour might suffice 
to record notes, but it hardly suffices to resurrect the 
continuity of thought of the previous day’s efforts. Starting up 
activities consume the same amount of time regardless of how 
little content is written; therefore, the more time one can spend 
writing at any one sitting, the greater the efficiency realized. 

Another method for shortening the starting up time is to 
end one day’s effort in the middle of a section—a paragraph, for 
example. Then, when writing resumes the following day, the 
writer has a framework in which to continue. Getting over the 
initial step of putting the first word to paper is the hardest part 
of writing. Invariably words flow after a bit of preliminary 
pump-priming and this priming should be done at the conclu¬ 
sion of each work session. 

Avoid interruptions 

Interruptions are not only annoying; they may be destruc¬ 
tive to a writer who is attempting to translate elusive concepts 
into written words. Some writers become “monastic” when 
they work. For example, they may retire to a solitary work 
space; they may take a telephone off the receiver; they may 
keep their office door closed; or, in a crowded office, they may 
arrange their desk so that they face a wall. These acts are not 
idiosyncrasies: they are methods which keep distractions to a 
minimum and thus increase efficiency. 
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Work at the same place 

Efficiency can also be achieved by working at the same 
place as much as possible. This ensures that most of the 
necessary materials will be available when and where they are 
needed. Furthermore, distractions of new environments are 
eliminated. 

Set daily goals 

Goal-setting is another method for increasing efficiency. For 
example, some writers set a daily goal of a specified number of 
words or a certain number of typewritten pages. Whatever 
technique is used is a technique for self-discipline, and discipline 
is essential for preparing an acceptable research product on 
time. 

Style 

Adherence to standards of grammatical usage is basic to 
effective style. But style-an almost indefinable quality of good 
writing—requires more than correctness. If written material 
“reads well,” if it communicates clearly, unambiguously, 
effortlessly, and concisely, then it has “good” style. In formal 
writing, the best style is usually the least obvious style. Good 
writing communicates concepts: it does not call the reader’s 
attention to syntax. Listed below are suggestions for writing 
effectively. 

Use the active rather than 
passive voice 

For example, “We conclude that...” is stronger, more 
forceful, and more direct than “It is believed that.” Or, “These 
data support the conclusion that. . . ,” is preferred to “The 
conclusions are supported by the following data.” The active 
voice is rigorous, less ambiguous, and invariably shorter. 

Use the appropriate 
“person" consistently 

For years a canon of scholarly writing required that the text 
be written in the third person. For example, with the exception 
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of the “rules” described in this section which are written in the 
second person, this entire text is written in third person. For 
certain types of writing, however, adherence to third person 
usage requires more words. For instance, it is more direct and 
shorter to write “We believe that...” In place of“It is believed 
that,” or “Now collate the data” in place of “The data should 
be collated now.” Although canons of writing (to say nothing 
of taste) require that the writer be kept in the background and 
that personal pronouns (I, we, you, me, my, our, and us) should 
not appear in the text of a scholarly paper, there are times when 
this rule should be bent for clarity, directness, and conciseness. 

Portions of Organization and Operations Manual cited 
earlier are written in the imperative rather than the declarative 
mode, and this also increases the directness and clarity. For 
example, the statement: “Make extensive use of headings and 
subheadings” is more direct than “Extensive use should be 
made of headings and subheadings.” 

Knowing when and how to deviate from the third person 
rule is a skill that must be acquired. For the sake of uniformity, 
however, the writer should use the same combination of person, 
voice, or mode consistently regardless of the combination 
chosen. 

Use simple sentences 

Nothing communicates so well as a simple sentence. 
Admittedly, excessive use of simple sentences may interfere 
with the “flow” of the discourse. But simple sentences are 
usually unambiguous, direct, and short. It is indeed strange that 
simple sentences are not used more often. 

Avoid modifiers 

Adverbs weaken sentences. Most of the time adverbs need 
not be used. Adjectives also tend to weaken sentences, but they 
are necessary at times. When subtle shades of distinction must 
be made, adverbs and adjectives should be used. Otherwise, it is 
better to write in nouns and verbs as William Strunk, Jr., 
advises. 
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Use orthodox language and 
expressions 

At the risk of suppressing individual creativity, it is 

r tu° mmended that onIy acce P ted spellings, terminology, and 
abbreviations be used. When in doubt about the extent to which 
an English word is acceptable, the writer should consult Evans 
and Evans’ A Dictionary of Contemporary American Usage, or 
Theodore M. Bernstein’s The Careful Writer. 1 Unorthodox 
spellings, obscure abbreviations, and excessive use of highly 
technical terminology reflect poorly on the writer. Worse yet, 
they obscure the message. 

Write at a level appropriate 
for the readers 

In order to write at a level suitable for the reader, the 
researcher must analyze his target audience. He must consider 
the audience s level of education, background, and familiarity 
with the subject matter. Then he must adapt his writing 
accordingly. If decisions are to be made on the basis of what is 
written, then the material must be organized and structured in a 
manner that facilitates decision making by people who are 
usually very busy. On the other hand, if the purpose of a report 
is simply to present new information, the style may be more 
discursive. In any formal writing, cliches, colloquialisms, and 
most idiomatic expressions have no place, regardless of the 
audience. 

Footnotes and Bibliographies 

There are two kinds of footnotes. The first kind of footnote 
expands or clarifies the main text. This kind of footnote is used 
when its inclusion in the main body of the text would break the 
continuity of the discourse or distract the reader. The second 
type of footnote cites the source reference. This type of 
footnote indicates the writer’s authority for making an assertion 


^Theodore M. Bernstein, The Careful Writer: A Modern Guide to 
English Usage (New York: Atheneum, 1965). 
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in the main body of the text. It would be used if a writer were 
paraphrasing or quoting another authority. In certain types of 
official reports, footnotes are also used to indicate non-concur¬ 
rence by a dissenting member of the group submitting the 
report. 

Footnotes provide a service to the reader. They indicate 
where additional information can be found, they identify the 
authorities whose observations and conclusions were cited, and 
they indicate that an assertion in the main body of the text 
reflects more than the writer’s personal opinions. In the latter 
sense, footnotes also increase the writer’s credibility. 

Notes are placed at the bottom of the page or at the end of 
a chapter or at the end of the text (but before the bibliog¬ 
raphy). Different organizations have different requirements for 
placing footnotes. For reader convenience, however, it is best to 
place notes at the bottom of the page. Constantly requiring the 
reader to turn to the back of the text is annoying. Not 
surprisingly, notes at the end of chapters or at the end of texts 
are rarely read except by diligent scholars. 

Although the form of a reference footnote may vary 
somewhat among different publishers and organizations, all 
reference footnotes contain essentially the same information: 
information relating to the authors), title, facts of publication 
(place, name of publisher, date), and volume and page numbers. 
For periodicals, the same information is required except that 
the place of publication is seldom mentioned. For specific 
information relating to the various forms of footnotes, Kate L. 
Turabian’s or William G. Campbell’s manuals should be con¬ 
sulted. These manuals have been basic reference guides for 
students for decades. 

Bibliographies usually list the sources that were consulted in 
preparing the report. In most cases many more works are used 
in preparing a report than are cited in a bibliography—works 
which every reader would assume were consulted by the author. 
Consequently, bibliographies typically list only the more 
relevant works bearing on the report. Any work that was 
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quoted or cited in a footnote, however, should be included in 
the bibliography. Bibliographies may also list books that the 
author recommends for additional information. Rarely do 
intelligence reports contain this type of bibliography, however. 

Like footnotes, bibliographic entries may take several 
forms, depending upon the requirements of the publisher or 
sponsoring organization. The appropriate style book should be 
consulted before the bibliographies are prepared. 

Bibliographic entries, unlike footnotes, must be organized 
or classified in some manner. One common method is to 
organize bibliographic entries as books, periodicals, and special 
reports. Within each group, specific entries might be arranged 
alphabetically by author, chronologically, or topically. 

Footnotes and bibliographies are services provided to the 
reader. They indicate where specific information can be found 
and how the reference can be located. Forms that footnotes and 
bibliographies take are of secondary importance: the critical 
point is that the entries be correct , complete , and that the 
forms are used consistently. It is for this reason that only one 
style manual should be used for any single report. 

Review and Revision 

Before a report is submitted, it should have undergone 
numerous readings and probably revisions as well. The writer 
typically reviews and revises as he composes. After setting aside 
a section of draft copy for several days (or weeks, if time 
permits), the writer may review the copy again. The writer who 
has any aesthetic sensitivity to prose typically is dismayed to 
find that the words he so carefully articulated earlier may now 
be incomprehensible. Thoughts may no longer hang together, 
the syntax may be garbled, assumptions may not have been 
stated, and all of the mechanical errors in the draft that were 
overlooked may become embarrassingly evident. The only 
solution, of course, is to wield the blue grease pencil, cut and 
paste vigorously, and review again. 
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One of the more difficult things for a writer to do is to 
submit his draft to a jury of peers. It takes courage to reveal 
one’s foibles to his colleagues. But as unpleasant as it may be, 
the step is absolutely necessary. Perspectives other than the 
writer’s are essential for uncovering hidden biases, unsupported 
assertions, and faulty logic. Few writers can uncover these 
things in their own material. 

How much should be reviewed and revised at one time 
depends upon the size of the work. If a book is being prepared, 
separate chapters would be logical portions to review and revise. 
If shorter reports are prepared, then an entire section of the 
report should be reviewed and revised at one time. Ideally, 
drafts should be reviewed as sections are completed. This 
precludes having to scrap the entire effort and starting over at a 
stage when time and funds may no longer be available. 

If the writer is able to employ or use the services of an 
editor, he is indeed lucky. Good editors can uncover errors and 
weaknesses that are not apparent to the writer. Good editors 
can improve a text immeasurably, provided that the writer 
heeds the advice they give. In the absence of editors, colleagues 
may suffice, but rarely do the researcher’s colleagues have the 
technical skills of a trained editor. 

Reviews often result in revisions, and revisions are time- 
consuming and exasperating. But they are essential for removing 
the inevitable errors—errors of commission and errors of 
omission. For example, spelling errors have an insidious way of 
occurring. “Difficult” words are usually checked by the writer 
and misspellings of these words rarely appear in finished texts. 
More common are misspelled familiar words, words which the 
writer may have assumed were correct and never bothered 
checking. A draft may be fairly “clean” until it is submitted for 
final preparation, and then “mechanical” errors may be 
introduced—typographical errors, for example. Fast readers who 
are reading familiar text are often unable to detect these 
“mechanical” errors. So, again, it is important for someone 
unfamiliar with the text to proofread the copy. 
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Rarely does a major report, thesis, or book make it through 
the entire publication cycle without some error remaining. 
Nevertheless, an error-free product should be the goal of every 
writer. The trivial flaw that escapes the writer’s and reviewer’s 
attention seemingly takes on greater significance when it is seen 
in its final printed form, and it remains a source of embarrass¬ 
ment as long as the work is read. 

Indispensable Aids 

Next to a typewriter, blue grease pencils, index cards, 
scissors and paste, and all of the other impedimenta for writing, 
a number of basic reference guides are indispensable for the 
writer. First, of course, is a reliable dictionary. If the researcher 
is a serious writer, he soon discovers that dictionaries vary 
widely in quality and in coverage, organization, and format. The 
specific edition chosen is largely a personal matter. Although 
most desk dictionaries will suffice for normal use, the serious 
writer will find that an unabridged dictionary is a necessity. 
Unabridged dictionaries point out subtle distinctions between 
nearly synonomous terms. This helps the writer to increase his 
precision. Obviously, unabridged dictionaries contain more 
entries as well. A dictionary should be found on every writer’s 
desk. More important, it should be consulted habitually 
whenever the slightest doubt as to the spelling or meaning of a 
word exists in the writer’s mind. 

In addition to standard dictionaries are dictionaries of 
technical terms. Technical dictionaries, dictionaries -of 
antonyms and synonyms, and a thesaurus are very useful aids 
for gaining precision in terminology and for increasing variety 
of expression. 

Even the most proficient writer encounters instances in 
which a subtle point of grammar must be clarified. For this 
purpose, any basic text in composition or expository writing 
can be used. One widely used work of this type is Porter G. 
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Perrin’s Writer's Guide and Index to English . ^ At times, a 
question may arise as to the acceptable usage of an expression 
or term. Three useful works for clarifying these points are Evans 
and Evans’ A Dictionary of Contemporary American Usage , 
Theodore M. Bernstein’s The Careful Writer (mentioned pre¬ 
viously), and Wilson FoWetCs Modern American Usage ^ 

Matters of style are treated extremely well in three works: 
William Strunk, Jr.’s and E. B. White’s The Elements of Style;^ 
Rudolf Flesch’s Art of Plain Talk; 1 1 and Barzun and Graff s The 
Modern Researcher , cited previously. Strunk and White’s short 
classic was originally prepared as a textbook by William Strunk, 
Jr. However, it is refreshingly direct, short, and not at all 
pedantic as many textbooks tend to be. Regardless of the 
researcher’s specialty this little book should be found on every 
writer’s desk. 

Rudolf Flesch has gained national recognition in his attempt 
to convince writers and speakers to say things simply. The 
change in many government publications reflects, in part, the 
success of this crusade. 1^ Finally, Barzun and Graff’s The 
Modern Researcher is one of the few works of its kind which 
lives up to the publisher’s “puff’: “the classic manual on all 

8 Porter G. Perrin, Writer's Guide and Index to English (Glenview, 
Illinois: Scott, Foresman & Co., 1972). 

^Wilson Follett, Modem American Usage: A Guide , Edited by Jacques 
Barzun (New York: Hill & Wang, Inc., 1966). 


1 ^William Strunk, Jr. and E. B. White, The Elements of Style (New 
York: The Macmillan Company, 1959). 

Rudolf Flesch, Art of. Plain Talk , (Riverside, New Jersey: Mac¬ 
millan Publishing Co., Inc., 1962). Related to this work is Art of Readable 
Writing (Riverside, New Jersey: Macmillan Publishing Co., Inc. 1962). 

1 ^Various editions of the Guide for Air Force Writing , for example, 
the 15 July 1969, edition, are excellent cases in point. 
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aspects of research and writing.” Although not prepared as a 
reference guide in the sense that Turabian’s manual and 
Campbell’s manual were prepared, the book can be used as a 
reference. Better yet, the book should be read by anyone 
performing research in any discipline. 

Summary 

t>Although the report preparation phase is the last phase in 
the sequence of activities in a typical research project, it need 
not be the last phase started. In fact, report preparation can 
begin as soon as data are available. 

t>As was the case in every preceding phase, prior planning is 
important to ensure that sufficient time will be available to 
produce a quality product. 

[>The form and content of a report should be tailored to the 
needs of the reader. If the report is a product of an assigned 
research project, then the formats prescribed by the agency or 
the assigning authority should be followed. If the report is a 
product of a self-initiated research effort, then the format 
should comply with established rules set forth in standard 
reference works, for example, the U.S. Government Style 
Manual . 

> Format and content apart, the manner in which a report is 
assembled is largely a matter of personal preference. The writer 
must determine what techniques and procedures work best for 
him, and then consistently and routinely adhere to his own 
standing operating procedures. 

t> Styles of writing vary widely. The best style for expository 
writing, however, is that which is least pretentious. Statements 
made clearly, concisely, and unequivocally characterize the 
better styles for reports. Accepted standards of grammatical 
use, a minimal use of modifiers, use of the active rather than 
passive voice, and use of orthodox terminology are recom¬ 
mended for scholarly, technical writing. 

£> Footnotes and bibliographies are usually present in formal 
research reports. Footnotes either cite the source or the 
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authority for an assertion made in the text, or they provide 
additional information to the reader. Footnotes and bibliogra¬ 
phies are aids to the reader and should contain sufficient 
information to enable the reader to locate the works for his 
own examination. Regardless of the form or format used in 
footnoting, it should be used consistently. 

C> Review and revision are continuous processes that occur 
during the entire time that a report is being prepared. However, 
prior to the submission of the draft for final publication, the 
draft should be reviewed critically by an editor (ideally), or by 
the writer’s peers as an alternative. Revisions should be made as 
sections are reviewed in order to preclude one’s having to 
rewrite the entire document when time may be extremely 
limited. 

>The written report is often the only tangible evidence of 
months or sometimes years of concentrated effort. The quality 
of the report reflects the quality of the research. Therefore, 
every attempt should be made to produce a high quality 
product. 
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CHAPTER XX 

AN EXAMPLE OF A SMALL-SCALE 
INTELLIGENCE STUDY 


Positive Unification Efforts Bearing Fruit After 27 Years 

Headline in The Korea Times , 

15 August 1972 

Earlier chapters described the sequence of phases through 
which an intelligence research program typically passes and 
described various methodologies that were appropriate for 
intelligence research. This chapter describes a short research 
study conducted by one of the authors in the Republic of 
Korea (ROK) in 1972. 

Not all of the steps described in earlier portions of the text 
were performed in this study. Because of constraints of time 
and resources and because the study was essentially self- 
initiated, the researchers had considerable freedom in defining 
the scope of the problem and in delineating terms of reference. 
Again, since the study was self-initiated, no review cycles were 
required. Unfortunately, nor was there any opportunity to 
influence the manner in which data were collected, as will be 
discussed later. 

The purpose of this brief example is not to cite an ideal 
example of how an intelligence research problem should be 
approached, but rather to describe how an actual study was 
conducted in light of operational constraints that intelligence 
researchers typically encounter. 

Background and Origin of 

the Problem 

Ever since the cessation of hostilities in the Korean War, a 
state of low intensity conflict existed between North Korea and 
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the Republic of Korea. The intensity of the conflict varied over 
the years and hit a peak during the 1968-69 period when North 
Korea dispatched a team to assassinate President Park Chung 
Hee, when a force of over a hundred armed North Koreans 
raided and terrorized the Samchok-Ulchin region of the 
Republic of Korea, when the Pueblo was captured, and when 
the EC-121 aircraft was downed. 

From 1969 to 1971, hostile acts committed by North 
Korean forces continued but the intensity and number of acts 
decreased. In the winter of 1971 and spring of 1972, North 
Korean diplomatic activities appeared to take on a new 
complexion. Although North Korean mass media still made 
vituperative attacks against the U.S., certain observers detected 
what appeared to be a conciliatory tone toward the ROK. On 
January 16, 1972, for example, an analyst of the 24th PSYOP 
Detachment in Seoul noted what appeared to be a departure 
from North Korea’s previous position on the subject of 
unification. 

Very briefly, the traditional stand of North Korea had been 
that a peace treaty between the North and South could be 
concluded only after the withdrawal of all U.S. forces from 
Korea. However, in mid-January, Korean Central News Agency 
(KCNA) wire service reports describing an interview that 
Premier Kim II-song had with a Japanese Socialist Party official 
suggested that this position had been reversed and that now a 
peace agreement between the North and South could be 
concluded without the precondition of U.S. forces’ actually 
being withdrawn. 

The implications of this reversal in policy—if it were 
true—were significant for not only North Korean—ROK rela¬ 
tionships but also for U.S. foreign policy vis-a-vis the ROK and 
North Korea. The perennial problem, of course, was deter¬ 
mining whether or not the appearance of reality had any further 
substance in fact. 

In its broadest terms, the research problem was defined as 
one of determining whether or not North Korea was changing 
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(or had changed) its previous position with regard to concluding 
a peace treaty with the Republic of Korea. 

Planning the Research Program: 

Problem Definition 

The small-scale research program that ensued was essentially 
self-initiated as stated earlier. Working closely with the 24th 
PSYOP Detachment on a larger basic research program con¬ 
tracted by the Department of Army, the researchers had neither 
time nor the resources to launch a thoroughly exhaustive study 
Furthermore, the researchers were constrained to work with the 
materials that were available or that could be provided or 
obtained without necessitating an extensive data collection 
effort. 

The problem definition phase ultimately resulted in the 
formulation of three hypotheses which will be described 
shortly. However, even before hypotheses could be formulated, 
an analysis of the consistency of North Korean policy state¬ 
ments as they appeared in North Korean mass media was made. 
This interim study had to be made in order to determine if the 
perceived change in North Korea’s stand on concluding a peace 
agreement as reflected in its mass media was real. This shorter, 
interim research effort will be described briefly because it 
involved many of the analytical steps described in earlier 
sections of the text. 

In order to determine if the apparent “change” in North 
Korean policy toward reunification was real or illusory, it was 
first necessary to determine how consistent North Korean 
public statements had been after Kim II-song’s interview with 
the Japanese Socialist Party leader had been reported. It should 
be noted at this point that very often the larger, overall research 
problem gives rise to smaller, more specific problems. In formal 
research programs, this “distillation” process is not at all 
uncommon and is a very important part of limiting the scope of 
a problem to manageable proportions, as discussed in Chapter 
IX. 
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In order to determine consistency in the public announce¬ 
ments by North Korea, all KCNA wire services from January 16 
to April 30, 1972, that were received through normal distribu¬ 
tion channels were reviewed. The alert reader who recalls the 
description of sampling in Chapter XVI will realize that this 
“sample” and the whole sampling process left much to be 
desired. For example, the wire service releases were provided by 
the Foreign Broadcast Information Service (FBIS). FBIS (at 
that time) monitored North Korean wire service transmissions 
on an ad hoc basis, according to an FBIS spokesman. If a 
transmission seemed important to the monitors, then it was 
transcribed; if the transmission was unrelated to FBIS’s moni¬ 
toring guidelines, then it was ignored. Consequently, the 
researchers had no idea of the number of wire service reports 
actually transmitted by KCNA, their frequency of transmission, 
or their periodicity. In short, the researchers had to use what 
they received. This condition is not unusual in intelligence 
research and analysis. 1 

Hundreds of wire service releases were examined of which 
seventy-eight pertained to peace proposals. This examination, it 
should be noted, involved both the exploratory and the 
investigative stage of data collection as described in Chapter 
XII. The wire service releases were those that were made 
between January 16 and April 30, 1972. These dates con¬ 
stituted a term of reference as described in Chapter IX. 

Having examined and identified every wire service release 
relating to the problem of unification, the researchers classified 
and coded the releases according to which one of four different 


1 North Korea had a relatively low priority compared with North 
Vietnam and the PRC during this period, and it is understandable that, 
given finite resources, FBIS could not respond to a sampling plan on a 
relatively low priority target. Thus, the sample of data used by the 
researchers was of very questionable representativeness. 
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peace proposals suggested by North Korea at various times from 
as early as April 12, 1971, to January 10, 1972, was addressed 
in the specific release. 2 The steps involved in the coding and 
classification procedure were similar to the steps described in 
Chapter XV. Categories were established, and data (wire service 
releases in this case) were assigned to the categories on the basis 
of “fit.” 

After the data were classified, hypotheses were tested 
relating to the consistency reflected among the wire service 
releases. Ignoring the details for the purpose of this brief 
description, it was concluded that the wire service releases were 
highly consistent in the manner in which peace proposals were 
addressed from the period of January 16 to April 30, 1972. 
Although earlier (and more restrictive) proposals were still 
mentioned, the more militant proposals were attributed to 
foreign spokesmen. The most recent proposal—the one that 
seemed to signal a change in policy—was mentioned most 
frequently by party cadre members. This conclusion became an 
input to the larger problem addressed in the project, the 
problem of determining if, in fact, North Korea had changed its 
previous policy regarding the conclusion of a truce agreement 
with the Republic of Korea. This was the primary focus of the 
study. 

Hypothesis Formulation 

On the basis of the analysis of the consistency of North 
Korea’s public statements pertaining to reunification (as well as 
other observables), three hypotheses relating to North Korea’s 
“peace offensive” were formulated. The formulation of the 
hypotheses involved inductive processes described in Chapter 
VIII and reiterated briefly in Chapter XIV. The hypotheses 
were as follows: 


2 North Korea had made a number of different proposals over the 
years. Most of the proposals were “hard line” and obviously stood little 
chance of being accepted by the ROK. The point of the short analysis was 
to determine if the proposals made after Kim 11-song’s new “epochal” 
proposal referenced any of the earlier hard line positions. 
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1) North Korea is attempting to reduce the hostile climate 
that existed on the peninsula for twenty-seven years. 
(For brevity’s sake, this will be referred to as the “peace 
hypothesis.”) 

2) North Korea is maintaining the former level of tension. 
(This will be referred to as the “business-as-usual 
hypothesis.”) 

3) North Korea is carrying out a plan of deception by 
making what purports to be a peace proposal merely as 
a guise for concealing hostile intentions. (This will be 
referred to as the “deception hypothesis.”) 

An examination of these three hypotheses shows that the 
third hypothesis was not “incompatible” with the first two 
hypotheses, even though it was “opposed” to them. Needless to 
say, these three hypotheses did not exhaust all possible 
hypotheses. 3 

Data Collection 

For the greater part, the researchers had neither the 
resources nor the authority to actually collect data. Instead, the 
researchers had to settle for data received through normal 
channels. In certain instances, data relating to incidents and 
conditions along the DMZ were provided to the researchers 
through J-2 channels. None of these data were in the form that 
required any detailed statistical analysis. 

Analysis and Findings 

. 

Rather than describing all of the data (“observables”) that 
were examined, a number of the. more significant events 


3 As a matter of interest, one of the researchers’ first tasks was to 
consider the range of possible hypotheses in order to establish a limited, 
workable number of plausible hypotheses. The reader will recall in Chapter 
XVIII that a minimum of three hypotheses should be postulated, as was 
the case in this instance. 
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examined will be described. Each major observable, or group of 
observables, was tested against each of the three hypotheses in 
order to determine under which hypothesis the observable was 
most plausible. (A Bayesian analysis as described in Chapter 
XVIII could also have been employed here.) 

1) The lengthy KCNA release on January 15, 1972, 
described an interview Kim 11-song had with the editorial 
committee of the Japanese newspaper, Yomiuri Shimbun, in 
which Kim answered some “questions” raised by the newsmen 
on January 10. Discussing the problem of reunification, Kim 
proposed that: 

... a peace agreement ... be concluded between the 
North and the South and the armed forces of North and 
South Korea be cut drastically under the condition 
where the U.S. imperialist aggressor troops are with¬ 
drawn from South Korea.'* 

It was not clear to the researchers what sequence of 
events was being proposed; e.g., would a peace agreement be 
concluded after U.S. forces were withdrawn or before? This 
ambiguity became significant in the further analysis because as 
the statement stood, any of the three hypotheses stated above 
seemed plausible. On the basis of this one input, the analysts 
hedged by giving equal weight to all three hypotheses. 

2) Beginning in the middle of January, KCNA made 
reference to the January 10 peace proposal almost daily. Some 
of the initial ambiguity in wording was resolved in subsequent 
“interviews” with Kim II-song and particularly in a meeting 
that Kim had with a Japan Socialist Party official, Kanji 
Kawasaki. It became quite clear that what was being said was 
that a peace agreement between the North and South could be 
concluded before U.S. forces were withdrawn. The 


^Lest there be any confusion about the dates, Kim II-song’s latest 
“peace proposal” was made during an interview purportedly held 10 
January. Issues raised at this interview were later clarified in another 
interview which was carried by KCNA on 15 January. This wire service 
release was received on or about 16 January by the 24th PSYOP 
Detachment analysts. 
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researchers were inclined to consider this “observable” as sup¬ 
porting the “peace hypothesis,” although the “deception 
hypothesis” was not completely ruled out. 

3) As was pointed out earlier, there was some question 
about the consistency with which North Korea treated the 
peace proposal issue. As analysis revealed, North Korea was 
highly consistent in its treatment of peace proposals, reverting 
only occasionally to a “hard line” stance and then attributing 
this stance to foreign spokesmen. This consistency supported 
the “peace hypothesis,” but again the “deception hypothesis” 
was not completely discounted. 

4) Analysis must consider the total situational context. In 
August 1971, Red Cross delegates from the North and South 
began meeting on a fairly regular basis to solve problems of 
relocating missing members of families and to solve other 
humanitarian problems. Initially, North Korea’s coverage of the 
talks (by KCNA) was characterized by severe criticism of the 
tactics of the Korean National Red Cross (KNRC-ROK) and by 
statements supporting their own position. After January 1972, 
however, the tone of North Korea’s coverage changed. Rarely 
was any direct criticism made of the KNRC, and the coverage 
became essentially factual and objective. The changing tone of 
KCNA’s coverage reflected the progress that the delegates were 
making. After months of disagreements and stalemates, an 
agenda was finally agreed to by both parties, and full-scale talks 
were scheduled. Interestingly, a “hot line” telephone linking the 
North and South Red Cross delegations was installed—a 
precursor of things to come, as it turned out. 

The toning down of vituperative attacks and the progress 
of the Red Cross meetings could not be separated, and together 
they strongly supported the “peace hypothesis.” The “deception 
hypothesis” was ranked second. 

5) Returning to the “peace proposals” again, by the 
middle of February whatever doubt the researchers had that 
what was being proposed by North Korea was a “new” proposal 
diminished rapidly as media from the Soviet Union and the 
People’s Republic of China praised the new “epochal” January 
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10 proposal. Furthermore, the U.S. press reported meetings that 
The Washington Post correspondent Selig Harrison had with 
North Korean spokesmen in Japan. Although “unofficial,” the 
input from Harrison was entirely plausible under the “peace 
hypothesis.” Again, the researchers opted for the “peace 
hypothesis” rather than the “business-as-usual” or the “decep¬ 
tion” hypotheses. 

6) Exposure that analysts have to the “other side” is 
generally vicarious through media. But others along the DMZ in 
Korea had more direct contact with “the other side.” There 
were two types of contact: one “symbolic” and the other real. 
The symbolic contacts were in the form of loudspeaker 
broadcasts which until early summer of 1972 rivaled the 
exchanges between Taiwan and the Chinese mainland for 
bitterness and abusiveness. The other contacts with the “other 
side” were engagements with armed infiltrators and agents. With 
respect to North Korean loudspeaker broadcasts, their tone 
changed noticeably in the spring of 1972. According to a 
statement made by ROK Defense Ministry, “ . . . Communist 
propaganda, usually full of abusive and slanderous words, 
showed a considerable change in contents as well as in 
expression.” In addition, the statement went on to say that the 
number of North Korean loudspeaker broadcasts declined from 
twenty-five a day to about sixteen broadcasts. This “observa¬ 
ble” refuted the “business-as-usual hypothesis” but supported 
the “peace hypothesis.” Naturally, the “deception hypothesis” 
still remained viable. 

7) North Korean loudspeakers were not the only medium 
whose output changed. A monitoring of KCNA output revealed 
that the number of wire service releases containing anti-ROK 
statements declined from a high of 27.4 percent in January to 
10.2 percent in April. This observable, like the preceding one, 
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also ruled out the “business-as-usual hypothesis,” but supported 
the “peace hypothesis.” 5 

8) Credibility of the word increases as the word is 
consistent with behavior. The researchers reasoned that if North 
Korea were in fact sincere in its purported “peace proposals,” 
aggressive behavior, hostile acts, and the like, committed by 
North Korean personnel should decline. The “observables” in 
this case are summarized in a ROK Defense Ministry statement 
which is quoted in part below: 

North Korea has infiltrated neither armed guerrillas nor 
agents into the Republic of Korea during the last seven 
months .... Not a single case of spy infiltration has 
been detected during the seven-month period beginning 
last December. 

The number of ceasefire violations on the part of North 
Korea was also reduced to 7,308 during the first six 
months this year. This compares with 7,900 reported 
during the corresponding period of last year. 

Since the beginning of this year, the Communists have 
committed 168 cases of border provocations, which 
represent a 50 percent decrease from the like period of 
last year.^ 

The “business-as-usual hypothesis” was refuted unequivo¬ 
cally by this “observable.” It was quite apparent that North 
Korea was not conducting its activities vis-a-vis the ROK in the 
same manner as it had in the past, The “deception hypothesis” 


determining this percentage reduction, as crude as the analysis may 
have been, involved some of the simple statistical techniques described in 
Chapter XVI. Determining whether the statements were hostile, neutral, or 
conciliatory involved the processes of coding and classification discussed in 
Chapter XV. 

6 Seoul Haptong, 0914 GMT 5 July 72 
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could not be excluded completely, but the complexity of the 
events militated against this hypothesis. In short, the hypothesis 
that was most strongly supported by the “observables” was the 
“peace hypothesis.” 

To summarize an analysis that took four months to 
perform, the researchers concluded that on the basis of extant 
observables (Figure XX-1), North Korea was in fact pursuing a 
“peace offensive” both in word and in deed. The “deception 
hypothesis” was not discounted, but it came in a very distant 
second. The “business-as-usual hypothesis” was discounted 
because too many observables directly refuted it. 

Validation 

A rare instance in which the conclusions of an intelligence 
researcher are corroborated occurred on July 4, 1972, when a 
seven-point communique was issued jointly by North Korea and 
the Republic of Korea. It stated, in part, an agreement on three 
principles of reunification; an agreement to refrain from 
“slandering and calumniating” the other side, and from commit¬ 
ting armed provocations; and, reminiscent of the earlier Red 
Cross talks, an agreement to install a permanent direct 
telephone link between Pyongyang and Seoul. In short, the 
peace hypothesis was dramatically supported by this announce¬ 
ment. 


Events that have transpired in Vietnam, Laos, and Cam¬ 
bodia since the time this short study was concluded have 
impacted significantly on both North Korea and the Republic 
of Korea. That the findings of this study have virtually no 
significance today epitomizes a notable characteristic of much 
intelligence research: it is highly perishable. 
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Figure XX-1. Evaluation of Observables in Light of Three Different 

Hypotheses 
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